Evaluation Gap

September 2004 – May 2006

Background

The Evaluation Gap Working Group was convened by the Global Health Policy Research Network, an initiative of the Center for Global Development, to address the problem of the lack of knowledge about the effectiveness of social programs in low- and middle-income countries.

Donors, developing country leaders and implementers of development programs face a dilemma: Everyone wants to ensure that donor and public monies are spent on programs that work, but measurement of impact is rare – and good quality measurement is rarer still. As a result, there is an evaluation gap. Program designers benefit little from accrued experience about what works, and developing country governments and their donor partners have little basis upon which to defend the wisdom of their investments or make adjustments if needed.

In part, this problem reflects the methodological difficulties of measuring some of the expected impacts of social programs, but experience shows that even when good methodologies exist, underlying disincentives to engage in rigorous evaluation reduce the likelihood that evaluations will be undertaken.

Approach

The central objective of the Evaluation Gap Working Group is to develop practical recommendations to solve this problem. The Working Group sought to: (a) understand the reasons for the lack of good impact evaluation, with a focus on health and education sectors; (b) the possible ways to make significant progress toward solving the problem. To do this, the Working Group:

1. Reviewed the current status of impact evaluation in social sector programs;
2. Explored the impediments to sustaining good impact evaluations;
3. Consulted with a wide range of stakeholders in developed and developing countries, including governments, multilateral and bilateral agencies, major international NGOs, the research community, and private foundations; and
4. Developed specific recommendations to address the problem that takes account of other complementary initiatives

The final report of the Working Group, When Will We Ever Learn? Improving Lives through Impact Evaluation, was launched in May 2006. Discussed by international leaders at a high-level meeting in Bellagio, Italy in June 2006, this report provided the impetus for the creation of an independent entity to sponsor rigorous impact evaluations of social programs in developing countries. CGD is now working to convene and provide technical assistance to a small group of committed developing country governments and donor agencies to serve as leading members who will design and launch an independent impact evaluation entity.

Funding for Working Group meetings, analytic work and consultations was provided under grants from the Bill & Melinda Gates Foundation and the William and Flora Hewlett Foundation.

Working Group Membership

Working closely with Nancy Birdsall and Ruth Levine of the Center for Global Development, the Working Group is led by William D. Savedoff, Senior Partner at Social Insight. The Working Group members include:

• Nancy Birdsall, President, Center for Global Development
• Francois Bourguignon, Chief Economist & Sr. Vice President, World Bank
• Esther Duflo, Associate Professor of Economics, MIT
• Paul Gertler, Professor of Economics, Haas School of Business
• Judith Gueron, Visiting Scholar, Russell Sage Foundation
• Indrani Gupta, Reader, Institute of Economic Growth
• Jean Pierre Habicht, Professor, Cornell University
• Dean Jamison, Senior Fellow, National Institutes of Health
• Dan Kress, Senior Program Officer, Bill & Melinda Gates Foundation
• Patience Kuruneri, Principal Social Sector Analyst, African Development Bank
• David Levine, Professor, Haas School of Business
• Ruth Levine, Senior Fellow, Center for Global Development
• Richard Manning, Chair, Development Assistance Committee
• Stephen Quick, Director, Inter-American Development Bank
• Blair Sachs, Program Officer, Bill & Melinda Gates Foundation
• William D. Savedoff, Senior Partner, Social Insight
• Rajiv Shah, Senior Policy Officer & Senior Economist, Bill & Melinda Gates Foundation
• Smita Singh, Special Advisor for Global Affairs, William & Flora Hewlett Foundation
• Miguel Szekely, Undersecretary for Planning and Evaluation, Ministry of Social Development of Mexico
• Cesar Victora, Professor, Universidade Federal de Pelotas
• Staff: Jessica Gottlieb, Program Coordinator, Center for Global Development

Members of the Working Group were invited to join in an individual capacity and on a voluntary basis.