This is a joint post with Rita Perakis.
It would be strange to try learning how to play music without listening to musicians. Similarly, learning about results-based aid programs requires listening to people who design and implement them. That is just what we did last week in a set of workshops about implementing programs that pay for results – programs which apply some or all of the principles that we’ve discussed here at the Center as Cash on Delivery Aid. As a result of discussing real experiences, we discovered that some of the challenges are quite different than we had anticipated while a number of common concerns have simply failed to materialize.
The Center convened these workshops to take advantage of a visit by Ellie Cockburn, economic adviser at the UK Department for International Development. Cockburn presented a series of pilot experiences that DFID calls “Payment by Results” and outlined some of the emerging lessons. Chris Berry, DFID head of profession for education, shared the lessons he has drawn from two results-based aid pilot education projects – one in Ethiopia and the other in Rwanda. Alan Gelb, CGD senior fellow, presented preliminary findings from research he is conducting on the World Bank’s 24 approved or proposed Program for Results operations. The discussion was also enriched by participants from environmental groups who explained Norway’s experience in paying for reductions in deforestation and from the Inter-American Development Bank which is paying for results in a Central American regional health program (Salud Mesoamérica 2015, described by Amanda Glassman here).
The discussion was focused on practical issues and challenges with designing and implementing results-based aid. We expected that results-based programs would involve more upfront costs than traditional aid programs because of the effort required to choose good outcome indicators and establish ways to independently verify them. We hadn’t anticipated, though, how difficult it would be for staff – in both funding agencies and recipient governments –to fully capture the differences between results-based aid and usual approaches. For example, partner governments in DFID’s pilot programs asked for guidance on spending even when the agreements gave them full discretion over the use of funds. If anything, this demonstrates that such RBA programs are not “business as usual,” requiring patience and persistence in getting the message across to staff on both sides of the aid relationship.
Meanwhile, the focus on measuring and paying for development outcomes has had a number of salutary effects, particularly for implementation. Recipient governments have told DFID that they value the way outcome measurements have helped them to manage their programs to improve performance. Salud Mesoamérica has experienced a similar pattern of improved implementation as a result of focusing both funders and implementers on outcomes.
We also saw that many concerns over COD Aid have not in fact materialized or were easily resolved. The lack of upfront funding for investments did not present a problem in any of the programs that we heard about – sometimes because the sector already had substantial foreign aid and domestic funding and sometimes because the funds were structured to reimburse countries for their local counterpart contributions. Concerns of losing “undisbursed funds” also seem to have been exaggerated because funders had contingency plans to reallocate unused funds to other sectors, to other countries, or to future time periods. None of the programs have encountered signs of perverse incentives or distorted behaviors as a result of basing payments on outcomes, which indicates that it is possible to mitigate such risks with well-designed outcome indicators and verification procedures.
Still, many questions remain. Participants highlighted the difficulty of knowing just how much to be “hands-on” or “hands-off”, how to provide technical assistance that is truly demand-driven, and how incentives work at different levels within a particular country. Many of the decisions that have to be made around results-based aid – like setting payment amounts or communicating results – are “more of an art than a science,” as Ellie Cockburn put it. Perhaps the key advice from the workshops is to understand context, consider a wide range of criteria, and be flexible without losing sight of the basic principles of a results-based approach. For our part, we consider these principles to include a clear focus on outcomes, recipient discretion, independent verification, and accountability of donors and recipients to their own citizens. Changing mindsets around how to deliver aid that embodies these principles is a tall order, but sharing these pilot experiences should speed up the learning process and the emergence of practices that realize the advantages of these new approaches.