International leaders meeting in Bellagio, Italy, last week committed their support for the creation of an independent entity to sponsor rigorous impact evaluations of social programs in developing countries. Meeting participants from developing countries, international organizations, bilateral donors, and private foundations said that such studies were crucial to learning what works in development and called for developing country governments to play a central role in the new entity.
The initiative is a response to a gaping lack of knowledge about which social development programs are effective. In 2005, donor countries committed $34 billion to aid projects addressing health, education and poverty in the developing world. Developing countries themselves spent hundreds of billions more on similar programs. Yet surprisingly few studies rigorously measure the impact of this spending.
Meeting participants agreed that a new entity - with members drawn from developing countries, bilateral and multilateral agencies, foundations, and NGOs - should be established to channel funds to high-quality, independent impact evaluations around key questions that confront policymakers in both donor agencies and developing country governments. These include, for example, evaluations of ways to increase the use of basic health services; to improve learning outcomes; to provide microcredit effectively, and others. Such evaluations would be done across multiple countries, to build a strong body of evidence from which generalized conclusions can be drawn.
The creation of the new entity was the central recommendation of the report of CGD's Closing the Evaluation Gap Working Group released on May 31, just ahead of the Bellagio meeting. (Video and transcripts of the event are now available on-line).
At the Rockefeller Foundation-supported Bellagio meeting, the Bill & Melinda Gates Foundation, the William and Flora Hewlett Foundation, India's Institute for Financial Management and Research, the African Monitor, and bilateral agencies from the United Kingdom, France, the Netherlands, Sweden, Canada and Australia all expressed interest in being involved in such an effort.
"Without impact evaluations that are rigorous, independent, and thus credible we cannot know what programs work. We cannot even argue convincingly that foreign aid itself works," said CGD president Nancy Birdsall. “By committing to do more impact evaluations, the organizations are joining together to learn--including learning about apparently good programs in health, education, and microfinance that might otherwise be scrapped for lack of evidence that they work."
Participants in the meeting reviewed the CGD working group report, When Will We Ever Learn? Improving Lives Through Impact Evaluation, which documented the limited evidence base for decision making. "The report reveals how little we really know about the effects of some very popular programs - HIV prevention, keeping girls in school, or community-based health insurance," said Ruth Levine, director of programs and senior fellow at CGD, and co-author of the report.
Dozens of prominent development practitioners, analysts, and officials have signed a statement of principles based on the report's recommendations urging the creation of an independent entity to foster more and better impact evaluations.
"Policymakers, researchers and program managers all know intimately how the lack of sound evidence about what works undermines program design and implementation, wasting resources and denying potential beneficiaries services that could have enabled them to improve their lives," said Bill Savedoff, a senior partner at Social Insight, an independent international consulting firm and co-author of the report. “Many in the development community agree that the time has come to fix this problem."
Participants in the meeting asked CGD to prepare for the launch of a new entity by convening a small group of committed developing country governments and donors to design it. CGD was asked to provide technical support while this group drafts a work plan and timetable for implementing activities to be undertaken during the next six months.
For more information, including a list of meeting participants, see the press release Donors Back Impact Evaluation of Programs in Developing Countries: New Initiative to Support Independent Studies to Determine What Works (PDF) also available en Espanol (PDF) and the Closing the Evaluation Gap initiative homepage.