Evaluation Gap Update | January 2009

"The question we ask today is not whether our government is too big or too small, but whether it works, whether it helps families find jobs at a decent wage, care they can afford, a retirement that is dignified. Where the answer is yes, we intend to move forward. Where the answer is no, programs will end. And those of us who manage the public's dollars will be held to account, to spend wisely, reform bad habits, and do our business in the light of day, because only then can we restore the vital trust between a people and their government." ~ U.S. President Barack Obama, January 20, 2009.

LEARNING WHAT WORKS

How can President Obama and other leaders know which public programs "work?" CGD's Evaluation Gap Working Group issued a report in 2006,When Will We Ever Learn? Improving Lives Through Impact Evaluation, addressing this very question. The Working Group recommended ways for aid agencies and governments to strengthen in-house evaluation capacity, and recommended establishing an independent international collaboration to promote more and better impact evaluations.

Subsequently, CGD helped a number of governments, private foundations and NGOs establish an independent organization, the International Initiative for Impact Evaluation (3ie), which has just recently issued its first calls for proposals. To hear about 3ie's activities, participate in its debates and get access to the latest impact evaluations, we encourage you to sign up for 3ie's periodic updates here.

WEIGH IN ON PRIORITY EVALUATION QUESTIONS BY JANUARY 30

What are the most important questions in development? The International Initiative for Impact Evaluation (3ie) has launched a survey to identify priority issues on which 3ie should commission impact studies to produce evidence for policy-makers in low and middle income countries. The questions listed in the survey were collected in an earlier exercise, but there is an opportunity to propose new questions. Take the survey.

Please forward this message to colleagues, and encourage them to also share it widely. The greater the response to the survey, the better. The survey will close on Friday, January 30.

COMMENT ON DFID EVALUATION POLICY BY MARCH 3

The UK's Department for International Development has as new draft evaluation policy and topic list out for public comment. The policy lays out the objectives and evaluation approaches that DFID will support in the future, including around more and better impact evaluations. The consultation process is an active one, and it's worth taking a few minutes to read the document and submit your thoughts about whether this is the right direction for DFID to pursue. Note that the deadline for comment is March 3. Full consultation documents, including instructions for submitting comments, can be found here.

NEW CGD PUBLICATIONS FEATURE IMPACT EVALUATION

In a recent CGD essay, "Toward Measuring the Impact of the World Food Program's Purchase for Progress Initiative," post-doctoral fellow Jenny Aker lays out the potential positive and negative effects of a new World Food Program initiative to purchase food commodities from local farmers to distribute within the same country or region. She questions some of the assumptions of the P4P, namely that farmers do not have access to markets, that establishing a parallel sales mechanism is an effective and sustainable means of increasing farm-gate prices, that such purchases will have a minimal impact on consumers’ prices, and that higher farm-gate prices in the short-term will serve as a sufficient incentive for farmers to increase production in the long-term. She outlines some of the potential unintended negative consequences of the program if not properly monitored, which are of particular concern in Sahelian countries with inelastic supply. Aker suggests that WFP and its donors measure its impact on a variety of groups in the short-and medium-term, in order to ensure that it is not doing (undue) harm and to identify the conditions under which it will work.

Alaka Holla and CGD non-resident fellow Michael Kremer survey evidence from recent randomized evaluations on the impact of price on access to health and education in developing countries in "Pricing and Access: Lessons from Randomized Evaluations in Education and Health." Randomized evaluations across a variety of settings suggest prices have a large impact on take-up of education and health products and services. While the direction of this effect is consistent with standard theories of human capital investment, a more detailed examination of the data suggests that it may be important to go beyond these models.

Closing the Evaluation Gap