Independent research & practical ideas for global prosperity

Evaluation Gap Update, November 2010

After our last newsletter mentioned Spain's evaluation agency, Eduardo Amadeo shared news of progress on institutionalizing evaluation in Argentina. Meanwhile, a debate over evaluating the Millennium Villages Project provides many perspectives on the feasibility and ethics of good impact evaluation. Fortunately, agencies like AusAID and DFID are stepping up to the plate by supporting 3ie's call for proposals for systematic reviews of important policy questions. All in all, I think it has been a year of progress for building better evidence for development policy. What's in store for next year? A new book on Impact Evaluation in Practice and maybe even a populist movement to demand peer-reviewed public policy (see Additional Resources below for more on these and other tidbits).


William D. Savedoff
Senior Fellow
Center for Global Development

Will Argentina be the next country to join the Evaluation Movement?

A recent article in La Nación reported that congressional representatives in Argentina, led by former social development minister Eduardo Amadeo, have introduced legislation to create an independent agency to evaluate public programs. (Amadeo actively contributed his time and energy to CGD's Evaluation Gap Initiative, participating in regional consultations in Mexico, Italy, and India). If successful, Argentina will join a handful of low- and middle-income countries with such agencies, including Mexico (CONEVAL), Colombia (SINERGIA) and Chile (DIPRES). The World Bank has videos and other resources that describe and assess the Colombian and Chilean experiences. India also considered establishing an independent evaluation office last year but no further information about that initiative's progress could be found. Looks like Argentina may beat them to the finish line on this one. (Photo: Eurico Zimbres)

Are Millennium Villages a good model for Development Programs?

In "When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages," Michael Clemens and Gabriel Demombynes show the limitations of current efforts to assess the Millennium Villages Project (MVP). They further argue that rigorous impact evaluations are not only feasible but also necessary if policymakers are going to make responsible decisions about increasing funding and expanding such programs. The authors discuss their paper in a CGD Wonkcast. Others have weighed in on the debate, including a response from the MVP group, a followup by Clemens and Demombynes, and blogs by Laura Freschi of Aid Watch, Michael Trucano, and Chris Blattman. (Photo: Millennium Village in Gumulira, Malawi, © UNDP)

In addition to doing more and better impact evaluations, the quality of existing studies needs to be assessed and the evidence sifted to determine just what we do and do not know about effective development programs. Fortunately, the Australian Agency for International Development (AusAID), the UK's Department for International Development (DFID) and the International Initiative for Impact Evaluation (3ie) have announced a joint call for proposals for such systematic reviews to summarize the policy lessons from the current evidence base while identifying gaps. Approximately 59 priority systematic review questions have been identified and can be found on 3ie's website along with application materials. The deadline for proposals is November 29, 2010.

In Quality of Official Development Assistance Assessment (QuODA), Nancy Birdsall and Homi Kharas propose to rank the effectiveness of aid agencies, including their commitment to promoting evaluation and learning. Aid agencies are assessed in terms of their efficiency, contribution to building institutions, reducing the burden of receiving aid, and transparency and learning. Evaluation comes to play in this final category. The proposed index uses data provided by 23 donor countries and more than 150 aid agencies. They explain their approach in this wonkcast.

Additional Resources