Outcome Mapping

Outcome Mapping is an approach that helps unpack an initiative’s theory of change and provides a framework to collect data on the immediate, basic changes that lead to longer, more transformative change. This allows for the plausible assessment of the initiative’s contribution to results.

Overview

Outcome Mapping focuses on understanding outcomes—the so-called ‘missing middle’ or ‘black box’ of results that emerge downstream from the initiative’s activities but upstream from longer-term economic, environmental, political, or demographic changes.

It can be used for planning, monitoring, and evaluating initiatives in order to bring about sustainable change. At the planning stage, the process of Outcome Mapping helps a project or program team be specific about the actors they intend to target, the changes they hope to see and the strategies appropriate to achieve these. For ongoing monitoring, Outcome Mapping can help to design and gather information on the results of the change process, measured in terms of the changes in behaviour, actions or relationships that can be influenced by the team or program.

Outcome Mapping can be adapted to a wide range of contexts. It enhances team and program understanding of change processes, improves the efficiency of achieving results and promotes realistic and accountable reporting.

Key concepts in Outcome Mapping

• Sphere of control: What can be directly controlled by an intervention – for example, use of resources and activities undertaken.
• Sphere of influence: What can be directly influenced but not controlled – for example, reactions to activities and subsequent changes in knowledge, skills, behaviour and attitudes of those who are not part of an organisation.
• Sphere of concern: The long-term changes that are sought but cannot be directly influenced by the intervention – but can be influenced through the actions of key actors outside the intervention.
• Outcomes: Defined as the behaviour changes of the people, groups or organisations within the sphere of influence of an intervention.
• Boundary partners: Key actors within the project’s sphere of influence whose changes in behaviours, relationships and actions with social actors outside the intervention’s sphere of influence are essential for achieving the intervention’s goals and vision and in whom resources and effort will be invested by the intervention to influence and support them. Examples include government officials, teachers, health workers, and youth leaders.
• Causal contribution: Recognising that multiple factors combine to produce a result, and therefore an intervention will never be the only reason for changes in boundary partners and social actors.

Principles for this approach

Three core premises of Outcome Harvesting are:

1. Social, policy & systems changes depend on changes in human behaviour
2. People contribute to their own wellbeing
3. Sustained improvements in people’s lives or environments depend on relationships

More recently, work on “OM+” (Outcomes Mapping + Equity, Gender, and Social Justice) has further developed the core principles as follows:

1. Equity-deserving people will contribute to their own well-being when they have the knowledge and skills to contribute and influence and make decisions.
2. Equitable social, policy, and systems changes depend on transformative changes in human behaviour.
3. Sustained improvements in people’s lives or environments depend on authentic relationships between diverse people, groups, and institutions (the actors) in the systems.

Five guiding practices for using Outcome Mapping to support transformational change are to:

1. Facilitate inclusive & equity-focused participatory change processes.
2. Be accountable to learning at individual, team and organisational levels.
3. Grow a complex adaptive system view & embrace uncertainty & experimentation.
4. Commit to iterative, collective sense-making with inductive & data-driven reasoning.
5. Lead from context and combine Outcome Mapping with other approaches as needed.

Steps in the process

Outcome Mapping involves 12 steps in four stages, which are briefly listed here: System Mapping, Intentional Design, Outcome and Performance Monitoring and Evaluation Planning.

System mapping

In recent years, System Mapping has been added as an explicit step at the beginning. This involves developing a contextually grounded picture of system actors, their roles, relationships, perspectives, and motivations.

Intentional design

Intentional design is based on seven components, which are usually developed in sequential order:

1. The vision: This describes the large-scale development changes that the organisation hopes to encourage.
2. The mission: This spells out how the organisation will contribute to the vision and is the ‘bite’ of the vision on which the organisation’s programme will focus.
3. The boundary partners: These are the individuals, groups, or organisations with whom the programme interacts directly and with whom it anticipates opportunities for influence – and who connect the program to its sphere of concern.
4. An outcome challenge statement: This describes the desired changes in the behaviour, relationships, activities, and actions (professional practices) of the boundary partner. It is the ideal behavioural change of each type of boundary partner for it to contribute to the ultimate goals (vision) of the programme;
5. Progress markers: A set of statements describing a gradual progression of changed behaviour in the boundary partner leading to the ideal outcome challenge. They are a core element in Outcome Mapping, and their strength rests in their utility as a set of desired changes that indicate progression towards the ideal outcome challenge and articulate the complexity of the change process. They represent the information that can be gathered to monitor partner achievements. Therefore, progress markers are central in the monitoring process. Progress markers can be seen as indicators in the sense that they are observable and measurable but differ from the conventional indicators used in Logical Framework Approach (LFA) in that progress markers can be adjusted during the implementation process, can include unintended results, do not describe a change in state and do not contain percentages or deadlines;
6. Strategy maps: These are a mix of different types of strategies used by the implementing team to contribute to and support the achievement of the desired changes at the level of the boundary partners. Outcome Mapping encourages the programme to identify strategies that are aimed directly at the boundary partner and those aimed at the environment in which the boundary partner operates.
7. Organisational practices: These explain how the implementing team will operate and organise itself to fulfil its mission. They are based on the idea that supporting change in boundary partners requires that the programme team itself be able to change and adapt as well, i.e., not only by being efficient and effective (operational capacities) but also by being relevant (adaptive capacities).

Outcome and performance monitoring

The monitoring stage involves four elements:

1. Monitoring priorities: provides a process for establishing the areas of the project to be monitored.
2. Outcome journals: a tool for collecting data about the progress markers over time.
3. Strategy journals: a tool for collecting data about the activities of a project.
4. Performance journals: for collecting data about organisational practices.

Evaluation planning

The evaluation stage involves one step:

An evaluation plan – which provides a process and a tool for designing an evaluation using Outcome Mapping. This can include clarifying whether and why an evaluation is needed, who the primary users would be, and what the evaluation should focus on. (Ambrose et al., n.d.a)

Causal Pathway Features

How this approach might be used to incorporate features of a causal pathways perspective

A causal pathways perspective on evaluation focuses on understanding how, why, and under what conditions change happens or has happened. It is used to understand the interconnected chains of causal links which lead to a range of outcomes and impacts. These causal pathways are likely to involve multiple actors, contributing factors, events and actions, not only the activities associated with the program, project or policy being evaluated or its stated objectives.

Outcome Mapping can be used in ways which incorporate the following features of a causal pathways perspective:

• Valuing actors’ narratives: Outcome Mapping pays attention to the experiences and values of people directly affected by an intervention or the system that an intervention is seeking to influence. Outcome Mapping encourages collective analysis, learning and sense making.
• Addressing power and inclusion: Outcome Mapping actively addresses power and inclusion by privileging the perspectives, experiences and leadership of people and communities who have been excluded, marginalised and discriminated against. “The approach values people as self-determining participants in processes to make their lives better.” (20 years of OM). A later development of Outcome Mapping, Outcome Mapping + described below which focuses on equity, gender, and social justice, more directly addresses equity, power and inclusion.
• Articulating explicit causal pathways: Outcome Mapping articulates explicit causal pathways, including intermediate outcomes, causal mechanisms, non-linear and plural (sometimes contested) pathways to change.
• Paying attention to a range of outcomes and impacts: Outcome Mapping attends to intended and unintended, as well as positive and negative outcomes.
• Using an iterative, bricolage approach to evaluation design: Outcome Mapping encourages the use of data and collective learning to inform subsequent data collection and analysis.
• Drawing on a range of causal inference strategies: Outcome Mapping draws on a range of causal inference strategies, especially inductive, collaborative and inclusive sense making processes and data collected in outcome, strategy and performance journals.
• Taking a complexity-appropriate approach to evaluation quality and rigour: Outcome Mapping collects data on observable change in outcomes, project implementation and organisational practices. The inclusive processes used in Outcome Mapping support responsiveness, reasoning and utilisation.

Background

History of this approach

The 2021 paper 20 years of Outcome Mapping Evolving practices for Transformative Change describes the history of OM:

"Outcome Mapping was first incubated by research organisations in West Africa and South-Asia in partnership with IDRC. It was developed by IDRC as an open-source method and toolkit for design, learning and evaluation practitioners and change makers around the world to use and adapt. In 2006, an online Outcome Mapping Learning Community of practice was formed, also with initial support from IDRC. Its purpose is to facilitate learning and knowledge sharing to help community members to collectively define and contribute to the changes they want to see in the world. Today, our diverse community has become an independent community-governed network with nearly 2000 members, from 127 countries, working on difficult social, political, and environmental problems." (Outcome mapping learning community, 2021)

"OM+ was developed to support [Outcome Mapping] practitioners who want to include a focus on equity, gender, and social justice as part of their approach. The thinking and practice of OM+ has evolved over the last seven years to provide practical direction for embedding core concepts of gender transformative practice and equity- Outcome Harvesting is a related evaluation approach which builds on the concepts of Outcome Mapping but is used during or after implementation to collect evidence of actual changes that have occurred and work backwards to assess plausible contribution." (Schaeffer & Zaveri, 2023)

Outcome Harvesting is a related evaluation approach which builds on the concepts of Outcome Mapping but is used during or after implementation to collect evidence of actual changes that have occurred and work backwards to assess plausible contribution.

Methods that are part of Outcome Mapping

BetterEvaluation defines an approach as a systematic package of methods. BetterEvaluation’s Rainbow Framework organises methods in terms of more than 30 tasks involved in planning, managing and conducting an evaluation. Some of the methods used in Outcome Mapping and the evaluation tasks they relate to are:

• Develop an evaluation design
  • Outcome Mapping uses a joint evaluation design to developing outcome challenges and progress markers together with boundary partners.
  • Outcome Mapping includes two generic planning tools: a monitoring plan for setting monitoring priorities (step 8) and the evaluation plan (step 12).
  • The 7 steps of the intentional design stage of Outcome Mapping provide users with a guided process for developing a theory of change that uses a combination of articulating mental models, existing documents, generic change theories, and group model building.
  • Outcome Mapping includes use of journals to record evidence of unintended changes in behaviour of crucial actors.
  • While Outcome Mapping can use a range of data collection methods, it does suggest the use of different types of journals for collecting data that can be used for monitoring and for evaluation..
  • Outcome Mapping provides a process and guidelines for continuous reflection among key actors involved in the initiative. By building in participation from the start, Outcome Mapping maximises the chances that findings will result in actual changes on the ground.

  Example

  The Agro-industry and Market Development Project for Arracacha

  "Organizations: Social and Economic Studies Institute, Bolivia; Ministry of Agriculture, Ecuador; School of Education and Health for Peasants, Peru; The International Potato Center (CIP) and the Consortium for the Sustainable Development of the Andean Region (CONDESDAN)

  Location: Coroico, Bolivia; San Jose de Minas, Ecuador; Cutervo, Peru

  The project objective was to support the cultivation of arracacha and processing it into rallado, a traditional sweet. A second goal was to strengthen local capacity to produce and market fresh and processed arracacha. Outcome Mapping was used to develop a monitoring framework for project activities in the three different countries. Due to the complexity of monitoring all the boundary partners, the team decided to select one boundary partner per country and each country chose a different type of partner. Journal reporting on each boundary partner was conducted every three months and was found to be particularly useful for project reporting to the donor agency.

  ‘Outcome mapping was used in tracking changes in behaviour of members of the agro-alimentary chain of arracacha producers, merchants and consumers. Use of this methodology was well suited to participatory management of the project and helped promote collective action based on the establishment of a shared vision and well-defined roles. Monitoring permitted a process of action/reflection, which allowed what didn’t work to be left behind, what was going well to be improved, and what was wrong to be corrected."

  Source: Raj, 2004.

  Advice for choosing OM

  What types of projects and programs would Outcome Mapping be appropriate for?

  Outcome Mapping is particularly appropriate for interventions where success depends on behavioural or social change and for programs operating in contexts where pathways to change are complex and unpredictable. This is because Outcome Mapping focuses on the intermediate outcomes that are within the sphere of influence of a program and how boundary partners can contribute to improved results further along the causal chain.

  What types of evaluations is Outcome Mapping appropriate for?

  Outcome Mapping can be used for intentional design, outcome and performance monitoring during implementation, and to inform choices about where to focus specific evaluations either during or at the completion of a project. Outcome Mapping provides a set of tools that can be used stand-alone or in combination with other planning, monitoring and evaluation systems to:

  • identify individuals, groups or organisations with whom the intervention will work directly to influence behavioural change;
  • plan and monitor behavioural change and the strategies to support those changes;
  • monitor internal practices of the project or program to remain effective;
  • create an evaluation framework to examine more precisely a particular issue.

  What level and type of resources are required for this approach?

  The Outcome Mapping FAQs provides advice on the type and level of resources needed:

  "For planning as a team, going through the first seven steps of OM usually takes three days. For monitoring, you should consider both the time to plan and to implement the monitoring system. If your monitoring is highly participatory, the amount of time it takes to create your monitoring system in a workshop is about 2-3 days. To implement the system will depend on how much you want to focus on documentation. The more focus on documentation and shared learning through participatory processes, the more time you can invest. The time put into an evaluation will depend on whether it is a self-assessment or an external evaluation.

  Garnering support for OM, clarifying accountability and learning needs and therefore prioritizing the data to be collected, identifying and using already existing spaces to conduct planning, monitoring and evaluation work, clarifying who your boundary partners are, and using skilled facilitation for the OM process, will make OM work more streamlined.

  The resources needed to use OM differ based on size of project, type of intervention, level of participation, monitoring and evaluation capacity of implementers, monitoring and evaluation system, and the type of data required, the level of analysis, and the uses. OM provides the tools to help plan what data gathering, analysis and use will look like, and therefore what resources will be needed. Approximately 3% of the total project/program budget is standard to estimate an amount for a monitoring and evaluation budget." (Outcome Mapping Learning Community, n.d.)

  How might this approach be part of an effective overall evaluation design?

  OM can contribute knowledge about how changes in the behaviour of important actors (key people, groups, organisations or institutions) have contributed to outcomes. Outcome Mapping has a focus on learning and complements a range of other evaluation approaches such as results-based management that evaluate expected outcomes.

  "In an Outcome Mapping Learning Community survey in 2019, practitioners reported combining Outcome Mapping with multiple methods and approaches to advance equity-focused and gender-transformative evaluation practices including: Outcome Harvesting, Contribution Analysis, Logical Framework, Most Significant Change, Social Network Mapping, Vulnerability Assessments, and Power and Interest Stakeholder Mapping."

  Advice for using Outcome Mapping effectively

  Potential users of Outcome Mapping should be aware that the methodology requires skilled facilitation as well as dedicated budget and time, which could mean support from higher levels within an organisation. Outcome Mapping also often requires a “mind shift” of personal and organisational paradigms or theories of social change.