Evaluation Gap Update

January 2014

Dear %%nickname%%:

Happy New Year everyone! This year is bound to have more and better impact evaluations published than ever before, reaping the investments in research made over the last decade. New evaluation organizations continue to be established and existing organizations are going through leadership changes. But there’s more work to be done. Where are all the negative results filed? When we can’t replicate a study, is it possible to have a civil discussion about it? Will all this work ultimately influence choices? And what might we know in December that we don’t know today?

Regards,


William D. Savedoff
Senior Fellow
Center for Global Development

Leadership change ahead for 3ie

Howard White has announced that he will be stepping down after six years as Executive Director of the International Initiative for Impact Evaluation (3ie). 3ie has flourished under White’s leadership in scale, quality and outreach as described in this blog by Bill Savedoff. If you know of anyone with the right profile – which means someone with the skills of a manager, a diplomat and a rigorous academic, along with energy, commitment, and humor – please encourage them to apply before February 3rd.

Germany steps up oversight of foreign assistance with new evaluation institute

Photo: Reuters

Germany's new independent evaluation institute for the country’s development policies, based in Bonn, is a year old. DEval has a mandate that looks similar to Britain’s Independent Commission for Aid Impact (discussed in a previous newsletter) because it will not only conduct its own evaluations but also help the Federal Parliament monitor the effectiveness of international assistance programs and policies. DEval’s 2013-2015 work program is ambitious and wide – ranging from specific studies of health programs in Rwanda to overviews of microfinance and studies regarding mitigation of climate change and aid for trade.

To err is human … to distinguish it from discrepancy is divine?

Photo: Flickr user: AASU Armstrong University Archives

In a recent Development Impact blog, Annette Brown and Benjamin Wood present thoughtful views on the way replication studies are reported. Brown and Wood refer to their own experiences with 3ie’s replication program and the controversy over Reinhart and Rogoff’s article on public debt. They recommend distinguishing “errors” (clearly-identified mistakes) from “discrepancies” (unexplained differences in results); and separating clear-cut methodological choices from those where researchers can legitimately disagree. They also remind us to pay attention to what’s important: "Whether in the original paper or in the replication study, we should focus … on what is relevant and robust."

Resources

• The 7th biennial African Evaluation Association (AfrEA) conference will take place in Yaoundé, Cameroon from March 3 -7, along with the annual meeting of The Network of Networks on Impact Evaluation (NONIE).
• In this vimeo, Lant Pritchett discusses whether impact evaluation is a useful tool for development effectiveness.
• A serious and humorous take on creating an evaluation culture from the Fieldstone Alliance.
• The Campbell Collaboration just posted "The Impact of Land Property Rights Interventions on Investment and Agricultural Productivity in Developing Countries: a Systematic Review."
• Watch J-PAL Executive Director Rachel Glennerster's presentation on Running Randomized Evaluations: A Practical Guide.
• Since 2002, the Journal of Negative Results in Biomedicine has been combatting publication bias.

If there is anything you think we should know about or would like to recommend for the next Evaluation Gap Update, please send it to [email protected].

Earlier issues of this newsletter are also on our website for your perusal.

Thanks to Albert Alwang and Kristina Wilson for help in putting together this newsletter.