Author: Mia Chung
Summary: Outcome Harvesting is a qualitative monitoring and evaluation methodology that supports internal learning, captures complexity, and reframes success. This blog highlights how ISET’s experience in developing monitoring systems and evaluations that are participatory, meaningful, and useful, using Outcome Harvesting.
Oh no, not more reporting…
We can’t deny the reputation – and frankly, reality – of much reporting for accountability or impact. So many reporting frameworks are single-purpose, overly simplified, and donor driven. The result – yet another seemingly unrelated requirement for teams funded for program delivery but over-burdened with administrative tasks.
At ISET, we take this complaint about monitoring and reporting very seriously. After all, our expertise is in monitoring, evaluation, research and learning, known variously as M&E, MERL, and MEL. In particular, we design monitoring systems and evaluations that are participatory, meaningful, and useful – including for those doing the reporting. After all, they know their programs best, and are invested in their improvement. Programs can only stand to benefit if staff are engaged in the review process, using data that is meaningful to them.
Creating a monitoring system with co-benefits
It has been ISET’s job since 2018 to develop and implement a robust, complexity-driven monitoring system for the Zurich Climate Resilience Alliance (the Alliance). For the design, we focused on the user, collaboratively developing products. Iterative process and training supported a shift from activity- to change-focused reporting, and in the process, added value for teams. We now use a mixed qualitative-quantitative system that centres the methodology of Outcome Harvesting. And, without a doubt, it is this portion of the system that Alliance teams have been the most – dare we say? – excited about.
Outcome Harvesting (OH) is a systematic but flexible, qualitative methodology for understanding progress and achievements in delivering complex goals, like social impact or systems change. OH is increasingly being used by programs big and small to monitor, evaluate, and/or learn about the complex changes they are contributing to and how those changes are being achieved. At ISET, we use it for monitoring, evaluation, and learning.
Outcome Harvesting involves capturing evidence of what changed and why, who contributed to those changes, and why the change is significant. These are crafted into “Outcome Statements”. When collaboratively used by analysts familiar with the context, Outcome Harvesting can support meaningful measurement and insightful analysis for many users.
Meaningful measurement
Complex programs, like ZCRA which is focussed on building climate resilience, are challenging to monitor and evaluate. Timelines may be lengthy before expected impact, counting losses avoided is difficult, and changes may be unexpected. M&E systems are commonly designed to capture only the limited metrics that donors working with short-term and simple programs expect to see. In contrast, a reporting system using Outcome Harvesting can capture substantive changes that reporting systems using traditional methodologies have difficulty with, such as those that are unplanned, difficult to measure, or unexpected. This yields more accurate evidence of success. Additionally, Outcome Harvesting starts with a change that has occurred and requires reflection on how it was achieved, better aligning with the flow of success. It can, for instance, highlight whether changes were expected or diverged towards opportunities that arose. Outcome Harvesting allows locally significant successes to be recognized alongside globally significant ones. Because meaning is made as reportees describe outcome significance and the contributions of the program and others involved, Outcome Harvesting takes context into account. Ultimately, OH data can be developed into well-evidenced narratives of complex change.
Insightful analysis for many different users
As MERL lead of ZCRA, we have been able to slice and dice data to track progress both against a theory of change as well as thematic priorities for donor accountability and internal learning. Using Power Query (a data engine) and Power BI (a data visualization software) together with Excel, we produce evidence-based visuals and a full database of success narratives that people across networks can draw from for different needs. Program teams have used it to validate quantitative indicators of success; program management has used it for strategic decision-making.
Click here for an example of how Outcome Harvesting analysis was used.
The value of Outcome Harvesting
Outcome Harvesting has proven useful for:
- Reflection and adaptive management
- Engagement with external stakeholders
- Proposal development
- Accountability reporting
- Informed strategic thinking on systems change
- Ad hoc knowledge and impact measurement insights
- Internal learning
- Externally sharing success stories
- Validating numeric impact reporting
The ISET approach
Ultimately, OH is a system for capturing narrative evidence of change as opposed to tracking activities alone. For many program teams focused on day-to-day delivery and success, reading the changes they have contributed to, gauged against a greater framework, can shift their perspective. It’s not that they don’t know what they’ve achieved, but reframing into a narrative allows teams to view their work in a different light. As one partner told us, “our teams were so proud to see their work considered a success”.
“Our teams were so proud to see their work considered a success”
With 20 years of proven success in systems thinking, application of resilience frameworks, climate change adaptation, and community-first global learning applied to local contexts, ISET can help you and your community track transformational impact. We are a registered BC non-profit, and committed to an equity informed, partnership and relationship focused way of working. We invite you to connect with us to talk about how you can use Outcome Harvesting to tell your story of success in building your community’s resilience.
Pingback: Building bridges to work together