January 29, 2010
Last week, I attended the board meeting of 3ie (International Initiative for Impact Evaluation), which took place during the Global Development Network’s 11th annual conference. While 3ie is quite new and the Board is working on its strategy and governance, the organization is clearly off to a good start. I was delighted to learn that the recent call for proposals garnered some $20 million worth of high quality proposals—twice the $10 million that 3ie has available to award. We need to know more about what works in development and now the international community has an organization capable of efficiently channeling funds into good quality policy-relevant studies.If you had asked me whether this would happen back in 2004 when Ruth Levine, Nancy Birdsall and I started the Evaluation Gap Initiative, I’m not sure I would have been so optimistic. In 2006, when the Working Group published its report “When Will We Ever Learn”, it sparked a lively and heated debate. Over time it became apparent to most observers that the substantive problem – the existence of an evaluation gap due to insufficient impact studies – was indeed real. The report’s two key recommendations have stood the test of time. First, the report called for international agencies to significantly increase their efforts to learn about the effectiveness of their programs. Second, it called for a collective effort through a new international organization to complement existing agencies by spurring evidence building in new ways.
Fast forward to today and the panorama is completely different (full disclosure: I’ve been kibitzing in this process as a paid consultant to 3ie while others do the heavy lifting!). The number and quality of impact evaluations being conducted by multilateral and bilateral development agencies, private foundations, and NGOs has increased substantially. Many more graduate programs are encouraging doctoral candidates to do applied research in developing countries. More importantly, policymakers all over the world are recognizing the value of impact evaluations and paying heed to them. For example, the U.S. Congress is considering legislation that would build a robust evaluation system for foreign aid – a directive that might be welcome to USAID’s newly appointed Administrator, Rajiv Shah, who was a member of the original Evaluation Gap Working Group.In terms of collective action, 20 countries, foundations and NGOs have joined to create 3ie, providing commitments of some $50 million over the next few years, principally from contributions of the Gates Foundation, Hewlett Foundation, and the UK’s Department for International Development. (It is time now for the other OECD countries to step up and meet the standard set by the UK!). In fact, the UK is demonstrating that building evidence through impact evaluation is valued across the political spectrum: current support for 3ie is under a Labor government while the Conservative Party’s recent policy green paper on development specifically calls for increasing the level of support to 3ie.
3ie’s first annual report describes remarkable progress in its first year under its Executive Director, Howard White, in terms of mobilizing funding, issuing grants, and outreach. I think it’s most critical achievement was to establish a robust and transparent grant review process. Yet, as Howard points out in the annual report, 3ie is not solely focused on promoting good research; rather its ultimate aim is “… to change policy to increase the impact of development spending.” And 3ie has already pushed ahead in this direction. For example, last year, the Indian government asked 3ie to help design a new independent evaluation office. The response included arranging a visit by Gonzalo Hernandez-Licona, who heads Mexico’s Consejo de Evaluación Nacional (Gonzalo also serves as a 3ie Board member). Other developing country governments are asking 3ie to assist them in assuring the quality of impact evaluations that they have commissioned. The newest call for proposals has boosted the likely relevance of future impact evaluation studies by encouraging direct engagement of implementation agencies in the study teams. When 3ie announces the grant winners, I think observers will be impressed by the range of questions being addressed and the caliber of engagement with policymakers.The Evaluation Gap initiative was by no means the sole source of all this activity. The World Bank, Inter-American Development Bank, and Poverty Action Lab, among others, were all upping the ante on impact evaluation work around the same time. Participating in last week’s 3ie Board meeting was a chance for me to see how the confluence of all this work has raised evidence building to a new level. I feel confident that we’re progressing to a time when, in those all-too-common-moments that a policymaker says to a staff person, “I need to know whether this will work and I need the answer tomorrow,” the basic research will be there to build better policies.For those who wish to follow the progress of 3ie directly—including receiving timely information on future calls for proposals—sign up directly for their newsletter at www.3ieimpact.org/mailing. Meanwhile, I’ll continue writing an occasional newsletter for CGD on impact evaluation issues more broadly. You can sign up for that here.
Fast forward to today and the panorama is completely different (full disclosure: I’ve been kibitzing in this process as a paid consultant to 3ie while others do the heavy lifting!). The number and quality of impact evaluations being conducted by multilateral and bilateral development agencies, private foundations, and NGOs has increased substantially. Many more graduate programs are encouraging doctoral candidates to do applied research in developing countries. More importantly, policymakers all over the world are recognizing the value of impact evaluations and paying heed to them. For example, the U.S. Congress is considering legislation that would build a robust evaluation system for foreign aid – a directive that might be welcome to USAID’s newly appointed Administrator, Rajiv Shah, who was a member of the original Evaluation Gap Working Group.In terms of collective action, 20 countries, foundations and NGOs have joined to create 3ie, providing commitments of some $50 million over the next few years, principally from contributions of the Gates Foundation, Hewlett Foundation, and the UK’s Department for International Development. (It is time now for the other OECD countries to step up and meet the standard set by the UK!). In fact, the UK is demonstrating that building evidence through impact evaluation is valued across the political spectrum: current support for 3ie is under a Labor government while the Conservative Party’s recent policy green paper on development specifically calls for increasing the level of support to 3ie.3ie’s first annual report describes remarkable progress in its first year under its Executive Director, Howard White, in terms of mobilizing funding, issuing grants, and outreach. I think it’s most critical achievement was to establish a robust and transparent grant review process. Yet, as Howard points out in the annual report, 3ie is not solely focused on promoting good research; rather its ultimate aim is “… to change policy to increase the impact of development spending.” And 3ie has already pushed ahead in this direction. For example, last year, the Indian government asked 3ie to help design a new independent evaluation office. The response included arranging a visit by Gonzalo Hernandez-Licona, who heads Mexico’s Consejo de Evaluación Nacional (Gonzalo also serves as a 3ie Board member). Other developing country governments are asking 3ie to assist them in assuring the quality of impact evaluations that they have commissioned. The newest call for proposals has boosted the likely relevance of future impact evaluation studies by encouraging direct engagement of implementation agencies in the study teams. When 3ie announces the grant winners, I think observers will be impressed by the range of questions being addressed and the caliber of engagement with policymakers.The Evaluation Gap initiative was by no means the sole source of all this activity. The World Bank, Inter-American Development Bank, and Poverty Action Lab, among others, were all upping the ante on impact evaluation work around the same time. Participating in last week’s 3ie Board meeting was a chance for me to see how the confluence of all this work has raised evidence building to a new level. I feel confident that we’re progressing to a time when, in those all-too-common-moments that a policymaker says to a staff person, “I need to know whether this will work and I need the answer tomorrow,” the basic research will be there to build better policies.For those who wish to follow the progress of 3ie directly—including receiving timely information on future calls for proposals—sign up directly for their newsletter at www.3ieimpact.org/mailing. Meanwhile, I’ll continue writing an occasional newsletter for CGD on impact evaluation issues more broadly. You can sign up for that here.Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.