Evaluation Gap Update
May 2006
In this issue:
Delhi Consultation on Impact Evaluation
On April 7, CGD convened a consultation on the Evaluation Gap initiative in Delhi with co-hosts Suman Bery from the 
National Council of Applied Economic Research and Rajat Gupta from McKinsey & Co., Inc. Meeting participants, including NGO heads, policy researchers, private sector leaders and officials in international agencies, agreed on the need to step-up impact evaluation with coordination and cross-learning, development of quality standards, and more (and more flexible) funding. Participants stressed the importance of engagement of potential users of evaluation results in the process of setting priorities; the value of building evaluation capacity in developing countries; and the need for "methodological pluralism" to address evaluation questions in different contexts. Participants endorsed a communiqué articulating a set of principles as a guide to action.
African Monitor Launched; Hosts South Africa Consultation on Impact Evaluation
The African Monitor, launched last week in South Africa, aims to act as a catalyst, bringing the experiences of Africans more fully into development debates. This new independent body raises key questions from an African perspective about the value and impact of development programs. A natural partner to the Evaluation Gap initiative, African Monitor founder and Archbishop of Cape Town, The Most Reverend Njongonkulu Ndungane, will co-host a consultation on impact evaluation in Cape Town on May 18-19. Policymakers and researchers from across the African continent will convene to discuss the role of African governments and civil society in motivating more and better impact evaluation of development programs.
Andrew Natsios and Carol Lancaster Debate USAID Re-Organization, Agree Evaluation Needs to be Strengthened
At a March CGD event, former USAID Administrator Andrew Natsios and former USAID Deputy Administrator Carol Lancaster disagreed on many points about the re-organization of US development assistance under the new "double-hatted" USAID Administrator, who is also Director of Development in the Department of State. But they did agree on one thing: evaluation of development programs is inadequate. Lancaster said, "Everybody does a miserable job of evaluation. I'm not talking about results-based management, which is not evaluation. I'm talking about figuring out whether we succeeded or not. We need an independent, capable evaluation function that is not associated with State aid or the particular agency that's trying to do evaluations. We need to figure out what we're doing right and what we're doing wrong." And Natsios agreed "all of the institutions are failing at that." He said, "The World Bank has problems with it, the Europeans have, we all talk about this privately. We don't get an objective analysis of what is really going on, whether the programs are working or not."
Canada and the UK Implementing National Impact Evaluations of Domestic Social Programs
Canada's Social Research and Demonstration Corporation pioneers large-scale demonstration projects using random assignment methods, with the aim of improving the evidence base for national social policymaking. Established in 1991 to assess the Self Sufficiency Project, SRDC rigorously evaluates social programs, including those designed to help poor people save, to encourage post-secondary schooling, and to explore community-employment innovations. The UK government, in partnership with the Policy Studies Institute and the US organization MDRC, has developed a demonstration project to test the effectiveness of interventions intended to improve job retention and advancement prospects for low-wage workers. Begun in 2003, the evaluation is scheduled to end in 2007.
Additional Resources on the Evaluation Gap
- A National Research Council report (pdf) on improving the role of science and technology in development assistance states that USAID "should recruit an adequate number of technically trained direct-hire employees to lead the design and evaluation of institution-building and innovation activities."
- A CGD Note highlights the link between evaluation and fighting corruption, noting that anti-corruption programs can - and should - be rigorously evaluated as the World Bank and others in the development community focus their attention on this problem.
- The Institue for Financial and Management Research and the Abdul Latif Jameel Poverty Action Lab at MIT will be running a 5 day course on how to conduct randomized impact evaluations in Chennai this July. The course is designed for development practitioners with preference given to applicants from the region. If you want more information about the India course, please contact Ritwik at ritwik@ifmr.ac.in; deadline for applying is May 15, 2006.
- Howard White of the World Bank's Independent Evaluation Group prepared this report for the DAC Evaluation Network Meeting in March. It highlights the CGD proposal for an independent impact evaluation initiative as one way forward to build more, needed support for impact evaluations internationally.
CGD will soon be launching the final report of the Evaluation Gap Working Group. Stay tuned for further details! If you have news or highlights to share in future mailings, please send them to Jessica Gottlieb (jgottlieb@cgdev.org).
Thanks and regards,
Ruth Levine
Director of Programs and Senior Fellow
Center for Global Development
