Payment by Results: One Size Doesn't Fit All

In recent years, donors have been making greater use of performance-based payment approaches to fund development programs. The UK Department for International Development, using the broader term being used across the UK government, has added “Payment by Results” (PbR) to the development lexicon. DFID has also stimulated the interest of its partners and contractors in PbR - whereby disbursements are made only after the achievement of agreed results - and it is signaling that it will be increasing its use of PbR programs.

Bond, the UK network of international development NGOs, is investigating what PbR means from an NGO perspective and last week released a report, Payment by Results: What it Means for NGOs .  The report makes a good contribution to the literature on results-based approaches because it looks at one subset of PbR approaches – contracts between development agencies and NGO service providers – and offers specific lessons from past experiences and guidelines for NGOs, as well as recommendations for donor agencies.

Because the Center for Global Development is viewed as an indefatigable advocate of PbR I wanted to highlight a few key points that the Bond report draws attention to, and clarify our point of view based on our work on approaches that fall under the large umbrella of Payment by Results such as Cash on Delivery Aid and Development Impact Bonds.  

All Payment by Results programs are not created equal.

Far from it. DFID’s PbR Strategy document, published last summer, acknowledges that there is no international definition of PbR and lots of different ways that programs can be designed within (at least) three broad categories of programs that fall under PbR: results-based financing or RBF (payments from funders or government to service providers), results-based aid or RBA (payments from funders to partner governments), and Development Impact Bonds, DIBs (programs that pay investors for the delivery of results). 

It is easy to think of PbR as one kind of aid program, or to confuse the alphabet soup of variations of PbR (RBA, RBF, COD, OBA, and P4R, to name a few). But the underlying theories, key features, lessons, and experiences of one approach can’t always be transferred to another. As Bill Savedoff says here, we often hear about approaches that are likened to COD Aid without paying attention to the difference between paying for activities or for outcomes. In this regard, the Bond paper does a good job of focusing on one particular category of PbR – payments from governments to NGOs for services – and of teasing out the implications of different kinds of payment triggers, shares of funding put “at risk,” and levels of the results chain at which results are defined. A PbR approach applied to NGOs is unlikely to share much in common with PbR programs focused on households or national governments. Similarly, programs in which 100 percent of funding is proportional to outputs are going to be very different from ones in which only a small portion of funding is linked to results. Each approach will have its own challenges, but, in trying to understand how and when PbR should be applied, it is not useful to treat all PbR approaches as though they are the same. 

PbR encompasses many theories of change. 

The Bond paper discusses how these design choices should affect an NGO’s decision whether to participate in a PbR program. What we have found is that the design features of a PbR program also reveal something else. They show that funders are confused about the theories of change embedded in their PbR programs. In a forthcoming paper, we distinguish four theories of change that are used by funders to justify and design their programs: (1) that the offer of a financial incentive will lead to some behavior change by the recipient; (2) that the performance funding makes results visible in a way that improves management; (3) that the focus on results will improve accountability to constituents or beneficiaries; or (4) that the agreement gives recipients more discretion and autonomy to innovate and adapt their activities. It is our sense that PbR programs are frequently criticized for relying on the financial incentive to change recipient behavior when most are actually designed to work through one of the other three channels.   

“Innovation” is not an automatic benefit of PbR.

At CGD, we have tended to focus on the fourth of these theories of change. We argue that giving ownership and responsibility to recipient countries creates space for learning, innovation, and long-term impact. The Bond paper makes a similar point about the value of flexibility in PbR contracts for NGOs … but the contract has to make that flexibility real. Our observation is that if funders pay for results instead of inputs, implementers can turn their attention to problem solving with their local knowledge and modify programs as needed to get the desired results – rather than focusing on satisfying the funder’s reporting requirements or sticking to an inflexible plan.   We have taken special interest in two forms of Payment by Results –  Cash on Delivery Aid and Development Impact Bonds – which both involve a funder paying for something at the highest possible level of the results chain (typically, a development outcome).  This matters because linking funds to the ultimate desired outcome, as opposed to specifically defined activities or even outputs, is what is expected to open up space for flexibility and innovation. (The Peterborough Social Impact Bond in the UK is one example of where we see this happening; the UK government is paying for the outcome of reduced instances of prisoner reoffending.)

But if funders think that providing discretion to the recipient is an important part of their PbR programs, we aren’t seeing much of it. Too often, funders are offering contracts that pay for results while simultaneously specifying the activities and approaches that must be used by the recipient. As the Bond report says, the risk is that implementers are attracted to PbR because of the expectation that it will give them flexibility and then become frustrated when the program’s design precludes this. Our research finds that very few programs are really testing the theory that PbR can improve results by creating greater scope for innovation because their designs continue to constrain recipient discretion. If flexibility, innovation, local ownership, and reduced administrative burdens are essential to the argument for PbR, then you can’t design PbR programs that simply add results payments on top of the requirements and restrictions of an input-funded program.

The point of any new financing approach is to try and get better results than the approaches used today. The question for CGD has always been, what kind of funding mechanism would actually make possible the kind of learning, adaptation, country ownership, accountability, and results focus that we know are important for development? Like the authors of the Bond paper, we believe that PbR “has a place in the portfolio of funding mechanisms available to tackle development problems,” and we think the cautions they present should be heeded and considered. But in terms of building evidence on the effectiveness of such programs, we will have to see PbR programs that actually make space for innovation before we will have experiences to assess and from which to learn.