Breakthrough to Policy Use
Reinvigorating Impact Evaluation for Global Development

Final Report of CGD's Working Group on New Evidence Tools for Policy Impact

Authored by Julia Kaufman, Amanda Glassman, Ruth Levine, and Janeen Madan Keller

Yagazie Emezi/Getty Images/Images of Empowerment

Yagazie Emezi/Getty Images/Images of Empowerment

Foreword

In 2006, when a CGD working group, led by Ruth Levine, Bill Savedoff, and Nancy Birdsall, published its report When Will We Ever Learn? Improving Lives Through Impact Evaluation, very few social programs benefitted from studies that could determine whether or not they actually make a difference. Since then, the world has seen tremendous progress in harnessing better evidence to inform public policy decision making, especially from impact evaluations of programs in low- and middle-income countries. But COVID-19 has put a spotlight on an unfinished agenda, underscoring the need for high-quality, timely, and context-specific evidence—for both effectiveness and political credibility of the response. The pandemic has demonstrated the cost in lives and livelihoods lost when policymakers make decisions based on incomplete or outdated evidence and data.

Given the potential real-world benefits, why have decision makers within governments, aid agencies, multilateral organizations, and NGOs not yet fully harnessed the value of evidence—including from impact evaluations—for better public policies? Looking ahead, how can the development community renew momentum and broaden bases of support for impact evaluation and the wider evidence agenda? These questions were the focus of a CGD Working Group on New Evidence Tools for Policy Impact, which set out to understand why these social benefits continue to go unrealized and to chart out a renewed funding agenda for greater value in government policymaking.

"How can the development community renew momentum and broaden bases of support for impact evaluation and the wider evidence agenda?"

The working group brought together a diverse group of policymakers and experts to review recent progress and identify outstanding challenges in the use and utility of evidence for global development, with a focus on impact evaluation. The working group’s final report highlights how far the field has come in addressing persistent critiques about the scale, generalizability, and policy utility of impact evaluation methods, reflecting a nuanced and more productive global conversation. It also offers recommendations on “what and how to fund to deliver on the promise of impact evaluation and bolster the broader evidence ecosystem” as two intertwined goals. The report urges governments and development partners to do more to integrate evidence and learning into routine operations and programming, while also emphasizing how researchers can elevate implementation, delivery, and cost analyses alongside impact evaluations for greater policy relevance. By spotlighting dozens of resources and examples of good practice and policy impact, the report helps ensure we are benefiting from—and not rehashing—well-developed contributions. In doing so, this report serves as a key resource for funders, practitioners, and students.

Across the recommendations, a core theme is the importance of shifting agenda-setting power and resources to those who best understand local policy contexts and priorities. One related recommendation is to advance equitable and lasting partnerships to support policy-immersed, but still independent, research and evidence groups. To this end, the report proposes a consortium to channel aid and philanthropic funding toward long-term, flexible institutional support for evidence organizations in lower-income countries, enabling them to move beyond short-term consultancies and time-limited projects towards sustained engagement with policymakers.

The working group’s final report is also timely within CGD as we celebrate our 20th anniversary. Since our founding in 2001, CGD’s research and analysis have centered on the need for rigorous evidence on development impact; what works to improve lives and health in low- and middle-income countries; and how to enhance the effectiveness of public and aid spending. CGD was built on the conviction that better and more open data, analytics, and evidence applied to policy can make a large difference for development outcomes. This focus has driven lasting changes in development policy and spurred major new initiatives, including the creation of 3ie—a leading provider and broker of impact evaluation evidence—following CGD’s Evaluation Gap Working Group.

The report offers a compelling case for greater and better investments in evidence-informed policymaking and in impact evaluation as an essential element of that effort. I look forward to seeing the partnerships, investments, and real-world impact that result from this CGD working group. I hope readers share my sense of reinvigorated commitment for the evidence agenda and translate that support into improved decision making for improved lives.

Masood Ahmed
President
Center for Global Development

Acknowledgements

The authors thank all working group members for their engagement and thoughtful comments, critiques, and suggestions throughout the working group process, which lasted from December 2020 to June 2022. The authors are especially grateful to Norma Altshuler, Chris Maloney, and Sarah Rose for their feedback. The report draws from working group discussions and background research conducted by CGD experts and commissioned from external partners. The project benefited immensely from members’ collective knowledge of policy processes and incentives, evaluation studies, and evidence resources and was further aided by consultations with nearly 100 policymakers and evidence experts from development institutions and the development research and evaluation communities. This work was also informed by a private roundtable discussion with philanthropic funders in October 2021. Many thanks are due to Emily Schabacker and Sarah Allen for supporting report production and Stephanie Donohoe, Jeremy Gaines, and Steven Love for designing digital media.

Note

This report was written by Julia Kaufman, Amanda Glassman, Ruth Levine, and Janeen Madan Keller and is a product of the Center for Global Development. Amanda Glassman and Ruth Levine served as co-chairs of the working group. The report’s content is based on the deliberations and input from working group members, background research, and consultations. The working group was composed of experts from low- and middle-income country governments, bilateral aid agencies, multilateral organizations, NGOs, and academia. All working group members participated in their individual capacities and have had the opportunity to review and provide input to this report. However, members do not necessarily endorse all components of this report, nor do its contents constitute a policy commitment by any party. All errors and omissions are those of the authors.

This working group was funded by the William & Flora Hewlett Foundation and individual CGD funders. CGD is an independent and nonpartisan research institution. There are no conditions or limitations on CGD’s independence in research, findings, conclusions, or resulting publications. Where appropriate, CGD may welcome and consider comments or views from funders, but CGD retains total discretion and final decision-making authority regarding program and project research topics, speakers, participants in activities, and report content.

Citation

Julia Kaufman, Amanda Glassman, Ruth Levine, and Janeen Madan Keller. Breakthrough to Policy Use: Reinvigorating Impact Evaluation for Global Development. Washington, DC: Center for Global Development, 2022.

Navigation