A Critical Moment for COD Aid or “The Trouble with Targets”

Nancy Birdsall and William Savedoff.CGD is following the growing number of development programs that experiment with results-based approaches, many of them influenced by the concept of COD Aid. The design stage is important to a successful COD Aid agreement but a critical moment occurs when the first results are reported. Will the parties honor the agreement as written and uphold it or renegotiate and undermine its integrity? In our last blog we discussed this critical moment in terms of finding ways to limit the pressures to disburse precipitously. In this blog, we explain how setting targets undermines the focus on actual performance and the ability to stick to an agreement.As mentioned in our last post, aid agencies are experimenting with programs that incorporate the main features of COD Aid: paying for outputs and outcomes, giving the recipient greater discretion to spend as they see fit, independent verification, and transparency. Once these results-based programs are up and running, they face a critical test when the first results are reported. In particular, most programs create expectations by setting annual targets and are then judged relative to those targets rather than to their baseline. And this means that even successful programs will be viewed as failures (a point also made in an earlier blog). By refusing to set targets, a results-based program can avoid this pitfall. How is it that targets can create such a problem?Here’s the issue. The core feature of a COD Aid agreement is a payment for each unit of progress. Following the example in our book, an aid agency might agree to pay a recipient government $200 for each additional child who completes primary school. There is no target – either in the first or subsequent years – just a payment for each additional child who graduates. If this were the entire agreement, then at the end of the first year – as long as the country achieves some progress – there will be a positive payment and an opportunity for a press conference where everyone is happy and congratulations go all around.  And so on in later years.If it were only that simple. Instead, most results-based programs are hybrids that create a payment for results within a legal agreement originally for traditional aid projects or budget support programs. This includes setting targets, even if only to prudently manage the budget as we discussed in our previous blog. So, on top of agreeing to pay $200 per primary school graduate, aid agencies and recipients typically set a target. For this example, the target might be 10,000 more children graduating from school, which implies an expected payment of $2 million at the end of the year. But stating this target is not neutral. It creates expectations which shape the way everyone will interpret the results revealed by the first progress report.In our example, the chances that the real result will be 10,000 are infinitesimally small and (given the track record of most development programs) odds are that the surprise will be on the downside. So the first year progress report will diverge from the target – up or down by a little or a lot – and the aid agency and recipient government will face a test. Do they stick to the core of the COD Aid agreement which pays $200 per graduate? Or do they try to pay out $2 million regardless of the results? Essentially, the target changes the way everyone interprets the outcome. To see why, consider what happens in different circumstances.The easy case is when the difference is small, defining “small” in relation to whether it upsets anyone. For example, imagine the report reveals that 9,782 students completed school or that 10,265 students completed school. With only about $50,000 at stake relative to the expectation of 10,000 children and $2 million, it should be simple for the parties to stick to the agreement. If actual results are less than the target, the recipient would have to accept somewhat less money than they had anticipated and the aid agency staff would have to explain the underspend to their managers. If the results exceed the target, the aid agency staff might find additional money to fulfill the per-student payment commitment even if it is more than the ex ante budget – and everyone is happy. However, if bureaucratic constraints prevail, the recipient will get the $2 million they expected and go home grumbling that they should have received more. Notice that even in this good circumstance, no one is focused on the amazing achievement of more than 9,000 students completing school or asking how it was achieved. Instead, the focus is on resolving the problem with the flow of money.A somewhat harder case ensues when the country succeeds and the difference is large. Imagine the country graduates 15,000 students - which should trigger a $3 million payment and far outstrip the budgeted amount. What then? Does the aid agency scrutinize the verification report to be sure the numbers aren’t being fudged? Does the recipient denounce the aid agency for withholding funds from a poor country? Do the parties negotiate to pay the additional amount with delay, once the aid agency comes up with the funds? Do the Finance Ministry, head of state, press and citizens focus on the money they should have gotten and ignore the important news about educational progress and how it was achieved?The hardest case occurs if the country’s progress falls far short of expectations. This is, incidentally, the most probable scenario since development programs systematically overestimate targets. Imagine the country graduates 3,000 students and the contract results in a payment of $600,000. The appropriate reaction should be for the recipient and aid agency to celebrate! Headlines: 3,000 more students completed school, the verification report functioned well, and $600,000 untied funds are transferred to the recipient. The recipient goes back to the drawing board and tries to figure out how to do better in the coming year. End of story?Not really. Notice that in the last paragraph we didn’t say anything about the targets. The framing created by the expectation that 10,000 children will graduate sets the stage for a different drama. Because of the target, the recipient expected $2 million and only received $600,000. They are likely to argue that failure to meet the target only shows how much they need the additional funds or that they were hindered by factors beyond their control. The aid agency is going to struggle with the “problem” of $1.4 million in unspent funds and the difficulty of explaining this supposed “failure” to their managers and government. But none of this would have occurred without the fictional creation of a target.This is why first year reports are such a critical moment for the success of a COD Aid program. If the parties renegotiate the agreement at this early stage, they are sending a clear public message that this is “aid business as usual.” They are declaring that the agreement only pretended to care about paying for results. Rather, the real goal all along had little to do with the number of children who graduated and everything to do with guaranteeing a flow of about $2 million from one country to another. Sometimes guaranteeing a flow of funding is a good idea, but, in that case, results-based payments aren’t the right instrument.The bottom line for results-based programs?

• Focus clearly on setting payments per unit of progress
• Stick to your initial agreement for at least three years
• Budget for surprises (bad as well as good), and
• Do not ever ever ever under any circumstances set a target – not even an implicit one!