USAID Announces Serious Commitment to Evaluation and Joins 3ie

Yesterday CGD hosted an event at which Raj Shah, marking his first year as USAID Administrator, gave a major speech to present the changes he has been introducing at USAID and to announce a new evaluation policy.  Shah said that USAID is “relentlessly focused on delivering results and learning from success and failure.” But you cannot learn without documenting what you’ve done. So Shah’s team, including his evaluation director Ruth Levine, have worked out a number of changes at USAID that represent a new and serious commitment to evaluation.A key aspect of the new policy is that it distinguishes between performance evaluations and impact evaluations. Performance evaluations will be required for all USAID projects and are primarily focused on monitoring whether activities have proceeded as planned. By contrast, impact evaluations are meant to be selective, focused on programs with approaches that are innovative or hold promise but which are not yet proven. Rigorous impact evaluations have been relatively rare among development agencies until this past decade, a problem highlighted in the final report of CGD’s Evaluation Gap Working Group, “When Will We Ever Learn?” (co-authored by myself, Ruth Levine and Nancy Birdsall on behalf of the working group). So, it is commendable that USAID’s new policy commits an average of 3% of program budgets to these impact analyses and consequently for learning which programs make a difference.However, I believe the most important part of the policy is its emphasis on rigor and independence in these evaluations. The new policy states that these impact evaluations will be conducted and led by external, independent researchers. While this does not guarantee an end to bias, it does introduce an important check on the natural impulse for agencies (and all of us for that matter) to soften or deny bad news.  If USAID follows through on this commitment to be strategic in selecting which projects get impact evaluations, provide adequate resources, pay attention to getting baseline data and good counterfactuals, and commissions independent researchers, it will be establishing a strong foundation for really learning about which approaches are effective. Then the challenge is to incorporate that hard-won knowledge into policies and decisions.In addition to this new evaluation policy, I was probably most excited by another part of Shah’s announcement: USAID will join and support the International Initiative for Impact Evaluation (3ie). In this increasingly global world, where we recognize that development projects are owned by the countries and beneficiaries and not by donors or development banks, supporting global efforts to produce evidence and learn is not only essential and goodhearted, but efficient and pragmatic. By becoming a member of 3ie, USAID joins an important partnership of countries and organizations, including Mexico, Uganda, Pakistan, Britain, Canada, Australia, Denmark, the Netherlands, the Millennium Challenge Corporation, the African Development Bank, the Bill & Melinda Gates Foundation, the William & Flora Hewlett Foundation, Save the Children USA and others who have collaborated in supporting this critical public good – the production of robust evidence regarding successful (and unsuccessful) development approaches.When CGD’s Evaluation Gap Working Group published its final report almost five years ago, it laid out two broad recommendations. First, it called on countries and agencies to improve their internal systems for generating and assimilating rigorous evidence from impact evaluations. Second, it called for contributions to and participation in a collective mechanism for building evidence globally, channeled through a new institution like 3ie. With this announcement, USAID has really stepped up to the plate. Congratulations to Raj and his team for bringing USAID into a leading role on this important global effort!