Evaluation Gap Update
May 23, 2014
We seem to be exposed to more information all the time, but is it relevant? Can we make sense of it? And how do we judge its quality? People are grappling with these questions in many ways. Some are proposing to assess programs by comparing them to a simple alternative – giving beneficiaries cash. Others are trying to make evaluation more useful to project implementers by giving them responsibility to do it. Then there are some who look to new statistical methods utilizing newly available and enormous datasets. There are plenty of ways to join the debate through upcoming conferences or even by designing your very own experiment online.
Regards,
William D. Savedoff
Senior Fellow
Center for Global Development
Cash: The new benchmark for foreign aid?
Cash transfer programs have been frequent subjects of impact evaluations. Using this evidence, researchers have debunked stereotypes, including claims that the poor spend their money on unsavory activities or are incapable of making the right decision to improve future earnings. Referring to these and other impact evaluations in a recent Foreign Affairs essay, Chris Blattman (Columbia University) and Paul Niehaus (co-founder of GiveDirectly) argue that cash transfers are successful enough to serve as a benchmark for assessing other kinds of aid programs. The paper inspired The Center for Global Development to host a lively debate on whether cash transfers are indeed “The New Benchmark for Foreign Aid.”
Source: Niehaus, presentation at CGD 2014
Institutions continuing to experiment: Where should evaluation functions reside?
Evaluation work at most agencies is typically delegated to operational departments so that studies will be more relevant and useful; then centralized to improve quality and strategic value; and then decentralized again when studies are considered out of touch. Illustrating this common pendulum swing, The United Kingdom’s Department for International Development (DFID) recently published a “Rapid Review of Embedding Evaluation” which assesses a 2010 policy to decentralize planning, commissioning and managing of evaluations. The UK’s Independent Commission for Aid Impact (ICAI) views this strategy from a different perspective in its report “How DFID Learns.” DFID is not the only agency to struggle with these challenges. “USAID Evaluation Policy: Year One” documented that agency’s effort to “embed” evaluations by giving its country missions responsibility for identifying and commissioning evaluation work.
The referenced media source is missing and needs to be re-embedded.
Adding “Big Data” to the toolkit?
Impact evaluations tend to rely on small datasets, directly collected from informants via interviews or questionnaires. Yet massive increases in available data and processing power have created a wave of “Big Data” analyses for all kinds of phenomena – from flu epidemics to terrorism – described by some as “A Revolution that will transform how we live, work and think.” In a recent Financial Times article, Tim Harford is not convinced. He writes “… while big data promise much to scientists, entrepreneurs and governments, they are doomed to disappoint us if we ignore some very familiar statistical lessons.” Fortunately, a leading statistician, Hal Varian, has provided an excellent overview to different uses of Big Data and a qualified sense of how it can be useful for hypothesis testing – the kind of research we try to do in impact evaluation.
Created using Tagxedo.com
Resources
If there is anything you think we should know about or would like to recommend for the next Evaluation Gap Update, please send it to [email protected].
Earlier issues of this newsletter are also available on our website for your perusal.
Thanks to Albert Alwang and Aaron King for help in putting together this newsletter.
|
