Rigorous Evaluation as a Practical Decision-Making Tool

Development Dialogue with IDinsight

IDinsight, a leader in using rigorous evaluation to transform development practice, presented at the Posner Center’s recent Development Dialogue. IDinsight’s Strategy and Internal Measurement Lead, Harlan Downs-Tepper, discussed creating an evidence-based mindset, beginning with a high-level discussion on when and why to evaluate, followed by case studies of IDinsight’s work in Zambia and Rwanda.

The talk was significant for the Posner Center Community, as the international development sector is under increasing pressure to prove impact, and this requirement has organizations rushing into collecting data and undertaking intensive evaluations, often without answering some fundamental questions before starting. As proponents of decision-focused evaluation, IDinsight presented the following decision-making tree:

This tool, while simple and straightforward, is a useful first step when assessing whether or not to proceed with an evaluation.

Harlan and participants then discussed areas of focus in an evaluation. Many organizations seek to capture as much data as they can possibly collect, rather than honing in on a defined domain where the data can more readily drive action and programmatic change. Harlan used a sample Theory of Change (ToC) to highlight a process for focusing an evaluation. Theories of change are causal mappings of the relationship between a particular effort and the change it seeks to bring about. He emphasized directing energy towards investigating the uncertain linkages and relying on existing literature and/or practice for places where causal linkages are better established.

To illustrate, he provided this diagram for an intervention involving insecticide-treated nets (ITNs):

In this example, the organization now has to focus their evaluation efforts on answering just that uncertain link within their ToC. Harlan encourages organizations, where feasible, to identify proximate outcomes with clear linkages to final desired outcomes. Proximate outcomes can often be more affordably and easily accessed.

Harlan then explored two specific case studies evaluations from IDinsight’s portfolio. The first example was with the Ministry of Health in Zambia, who wanted to understand if simple ‘mama kits’ with a few basic supplies for mothers of babies would enhance in-hospital deliveries. IDinsight conducted a randomized control trial, ultimately learning that this simple kit (valued at $4 USD) was a sufficient enough nudge to increase delivery in facilities by 47%. The ministry of health has taken this data and scaled the program nationally. To learn more, see their policy brief here.

The next case study centered on a higher education program, Kepler, which is a blended learning model that combines online coursework with in-person coaching and support services to provide Rwandans access to a US-accredited degree. Kepler wanted to understand the efficacy of their model relative to other higher education institutions in Rwanda before working to scale across Sub-Saharan Africa. IDinsight’s impact evaluation revealed that Kepler’s model did improve student performance and they are translating these results into increased fundraising and an effort to scale the model. To read the full report, go here.

We’re appreciative to IDinsight for sharing their deep expertise with our community, and for the technology in our IEEE Global Classroom for making it possible to bring Harlan here virtually.

Stay tuned for future opportunities at the Posner Center to learn from leaders in evidence-based development.