The Evaluation Gap Working Group was comprised of the following members who served in their individual capacity and not as representatives of their respective institutions:
- Nancy Birdsall, President, Center for Global Development;
- Francois Bourguignon, Chief Economist & Sr. Vice President, World Bank;
- Esther Duflo, Professor of Economics, MIT;
- Paul Gertler, Professor of Economics, Haas School of Business;
- Judith Gueron, Visiting Scholar, Russell Sage Foundation;
- Indrani Gupta, Reader, Institute of Economic Growth:
- Jean-Pierre Habicht, Professor, Cornell University;
- Dean Jamison, Senior Fellow, National Institutes of Health;
- Patience Kuruneri, Senior Policy Analyst, African Development Bank;
- Ruth Levine, Senior Fellow, Center for Global Development;
- Richard Manning, Chair, Development Assistance Committee;
- Stephen Quick, Director, Inter-American Development Bank;
- William D. Savedoff, Senior Partner, Social Insight;
- Raj Shah, Senior Policy Officer & Senior Economist, Bill & Melinda Gates Foundation;
- Smita Singh, Special Advisor for Global Affairs, William & Flora Hewlett Foundation;
- Miguel Szekely, Undersecretary for Planning and Evaluation, Ministry of Social Development of Mexico;
- Cesar Victora, Professor, Universidade Federal de Pelotas.
The major areas of agreement among the Working Group members were:
- Too few good quality impact studies are being conducted and made available
- Too much money is eing wasted on poorly done studies
- Impact evaluation is best done in collaboration between researchers with knowledge of evaluation methods, executing agencies, and project designers and interacting with policymakers and stakeholders.
- Credibility of studies does not depend so much on being produced by "independent” institutions, rather credibility depends on the quality of the work, its interpretation and its presentation
- The knowledge produced by impact evaluations is a "public good" (in the technical sense)
- Creating incentives to demand impact evaluations would be more effective than improving the supply of impact evaluations
- Existing interest in impact evaluations could be leveraged if funds and technical support were more readily available
- Experimental methods should be used where possible, but they are not appropriate for all questions or contexts
- No single initiative will fully address the Evaluation Gap
- Preference for non-bureaucratic solutions
- Preference for engagement and commitment of major international agencies and governments
A major area of debate was whether the EGWG should indicate a strong preference for randomized control studies. The final deliberations concluded that if the Working Group's mandate were to address issues related to all kinds of evaluation, focused attention on randomized control studies would not be warranted. However, the Working Group's mandate was to focus on impact evaluations and within this category of studies it is apparent that randomized control studies hold significant promise for advancing knowledge, that they are not currently being conducted in sufficient numbers, and that other initiatives are not adequately addressing the need to promote such studies.
Once this was resolved, conclusions were quickly reached on other areas of disagreement. Given the focus on improving impact evaluation by promoting some additional, well-done, impact evaluations – preferably through supporting randomized control studies, it became apparent that earmarking funds and managing them through a new and independent agency would be the best way to proceed. Hence, the decision to recommend creation of the International Impact Evaluation Consortium.
The following individuals were interviewed or provided comments to the EGWG:
- Raj Shah (BMGF)
- Ruth Levine (CGD)
- Catherine Cameron, Consultant, DFID & Agulhas, Inc.
- Max Pulgar-Vidal, Special Advisor, Office of Development Effectiveness, IDB
- Mayra Buvinic, Division Chief, Social Program Division, IDB
- Inder Ruprah, Senior Economist, Office of Evaluation, IDB
- Eduardo Lora, Senior Economist, Research Department, IDB
- Charles Griffin, World Bank
- Gregory Ingram, Director, Office of Evaluation & Development, World Bank
- Charles Sherman, National Institute of Health (NIH)
- Carol Peasely, Counsellor, USAID
- Carol Lancaster, Prof. GWU (formerly with USAID)
- Patrick Kelley, Director, IOM Board on Global Health
- Eduardo Gonzalez Pier, Ministry of Health, Mexico
- Paul Gertler, Professor, Berkeley
- Jeremy Hurst, OECD
- Julio Frenk, Minister of Health, Mexico
- Stephano Bertozzi, Berkeley & Institute of Public Health (Mexico)
- Ricardo Hausman, Professor, Kennedy School
- Tom Bossert, Professor, HSPH
- Rachel Glennerster, Director, Poverty Action Lab, MIT
- Abhijit Banerjee, Professor, Poverty Action Lab, MIT
- Bernhard Schwartlander, GFATM
- Richard Feachem, GFATM
- Elizabeth Docteur, OECD
- Daniel Klagerman, Ministry of Economy, France
- Richard Manning, DAC
- Hans Lundgren, DAC
- Paul Delay, UNAIDS
- Ties Boerma, WHO (and Health Metrics)
- Francois Bourguignon, Chief Economist, World Bank
- Stephen Quick, Manager, Office of Evaluation, IADB
- Jim Heiby & Karen Cavanaugh, USAID
- Calestous Juma, Harvard
- Rob D. van den Berg, Global Environment Facility
- Michael Schroll, WHO
- Binh Nguyen, Asian Development Bank
- Ariel Fiszbein, World Bank
- Coralie Gevers, World Bank
- Owen Barder, Center for Global Development
- Neils Dabelstein, DANIDA
The EGWG process and findings were discussed at the following meetings:
Health Metrics Network. Meeting at Johns Hopkins University, Baltimore, MD. May, 2004.
VII Meetings of the LACEA / IADB / WB Research Network on Inequality and Poverty. San Jose, Costa Rica. November 3, 2004.
World Health Organization. Staff involved in GAVI and Health Metrics Network. Nov. 9, 2004. Geneva, Switzerland.
Global Fund for Aids TB and Malaria. Nov. 8, 2004. Geneva, Switzerland.
Development Assistance Committee Evaluation Network. Nov. 10, 2004. Paris.
2ème conférence AFD / EUDN. Aide au développement: Pourquoi et Comment. Quelles stratégies pour quelle efficacité? No