GAO Report on Vanuatu: A Case of He Said/She Said and What Should Have Been Said

*This is a joint posting by Sheila Herrling and Sarah Rose
On July 26, 2007, the House Foreign Affairs Subcommittee on Asia, the Pacific, and the Global Environment held a hearing on a GAO report that claimed the Millennium Challenge Corporation overstated in its public documents the projected impact – in terms of per capita income, GDP growth and number of beneficiaries -- of its $65.7 million compact with Vanuatu. The MCC, in testimony, maintained that its projections were factually correct and never intended to mislead, although it could see the possibility for misinterpretation.
After much “he said/she said” between GAO’s David Gootnick and MCC’s Rodney Bent, subcommittee members seemed satisfied to chalk it all up to “a difference in bookkeeping.” Not so satisfying to Gootnick, whose bottom line was the lack of transparency with which the MCC portrayed the compact’s projected benefits, which becomes an issue of credibility.
Folks should read the GAO report and MCC’s testimony (or watch the video on the subcommittee website) and form their own opinions – and share them with us here on the blog. But here’s just a quick summary of some of the most important he said/she said (note that the paraphrasing is ours, but it is intended to break out all the diplomatic language and boil it down to the core messages debated at the hearing):
GAO said: The MCC projects it will increase per capita income by 15% in 2010 and 37% by 2015. This is an overstatement because these estimates reflect cumulative levels, not annual levels, which would be closer to 4-5% at 2010 and 2015.
MCC said: Our analysis is factually correct, but we see how readers could get confused. We’re sorry, it wasn’t intentional, and we’ll be more precise from now on.
GAO said: MCC’s public documentation looks as though the compact is projected to contribute to an additional 3% GDP growth per year. This is misleading since Vanuatu’s GDP growth will more likely (after a year of 6% growth in 2007) remain around 3% with or without the compact. It would have been more accurate for the MCC to state that the compact is projected to increase the level of GDP by 3% and avoid vague wording that could be interpreted as a growth impact.
MCC said: GAO misinterpreted our portrayal, as any “reasonable reader” might do. We always meant to portray that Vanuatu’s GDP would be perpetually three percent higher with the compact than without and that the compact would have a moderate but rising effect on the per capita CGD growth rate, albeit much smaller than three percentage points.
GAO said: MCC’s documentation states that the beneficiaries of the compact will be 65,000 rural poor when actually only 43% of the compacts benefits will accrue to these rural poor. The remaining 57% benefit tourism and transport companies and other local businesses.
MCC said: We are confident that are numbers are credible, based on best available information which often times is very old. More broadly, a focus on reducing poverty through economic growth inevitably means that households above the poverty line (and many are not far above) will also benefit from economic growth.
GAO said: MCC failed to properly account for key risks in its calculations, such as: sufficient cost contingencies to cover potential cost overruns; more accurate (slower) phasing of compact benefits; Vanuatu’s historically weak maintenance record (higher risk over the longer-term than calculated by MCC); and the potential beneficiaries’ (uncertain) response to new opportunities generated by the projects (efficiency gains from the roads and bridges may not translate into economic gains).
MCC said: We just chose different risks than GAO, but believe they sufficiently capture a credible threshold and we will rigorously monitor progress and adapt to any unanticipated events.
So, you can see why the subcommittee wanted to chalk it up to “a difference in bookkeeping” or “a difference in interpretation.” Here’s the dilemma: the issue GAO raises on transparency is a serious one, and we sincerely hope that when the GAO completes its review of the other 12 compacts it doesn’t find that the MCC consistently chooses a less than straightforward portrayal of its projected impact. On the other hand, all parties agree that the projects funded by the compact are the right ones for growth and poverty reduction; that the MCC is way ahead of other donor agencies on transparency of its selection and evaluation indicators; and that there is pressure from all sides for the MCC to show implementation results. We think Congresswoman Watson (D- CA) hit the nail on the head when she said, “we need quality data at the outset and impact evaluation at the end.”
If the MCC overstated the projected impact of its investments in Vanuatu, and Congress, the MCC Board and American taxpayers are serious about the accountability-for-results component that grounds the MCA program, then the biggest losers will be the MCC and Vanuatu if their partnership fails to deliver its stated results. Even if the Vanuautu compact achieves significant results, if MCC’s estimated target for impact was inflated, the project will appear to have failed. So, where does this leave us? Where should Congress and the public focus their oversight and energy?
Sheila and Sarah said: Somewhere between “let’s chalk it up to a difference of bookkeeping” and Congressman Lantos’ press release on the GAO report’s findings that indicts the MCA (talk about overstatements!) are important questions for the MCA and MCA-watchers to tackle. First, transparency and presentation of facts matter. The MCC may not have intended to mislead but it did. One quick and easy remedy would be for the MCC to make public its country Investment Memos which contain the deeper analysis and justification for funding. Much like the World Bank and IMF make public their staff appraisal documents, the MCC should follow suit. Second, efforts redirected from implementing and measuring the progress and impact of the programs on the ground to reassessing the math and presentation of the projected impact is not the best use of scarce staff time and resources. Rather, the MCC’s feet should be held to the fire at the end of the compact for delivering the results they initially said they would (allowing, of course, for adjustments when new baseline data become available during the course of the compact). If they and the country under-perform relative to expected results, another compact should not be forthcoming, hence the incentive for straight-forward and realistic portrayal of impact benchmarks from the outset. Recalculating and/or re-portraying in a new way the originally calculated impact to more positively portray the end results should be rejected.
Second, good and timely data is hugely important. Without it, countries and donors cannot credibly measure the impact of their interventions. The MCC is substantively contributing to data collection and monitoring and evaluation practices in each of its compact countries. The question is: do they do it early enough? Should we be factoring the availability of good and timely data into the MCA selection process and using more of MCA’s 609(g) resources to help countries improve their data before compact development?
Third, it’s time to define the word “transformational.” The Vanuatu case provides good context for addressing this issue. Does the compact “transform” the economy of Vanuatu? It certainly adds value. It funds infrastructure that is critical to building a strong economy and would have gone unfunded without the MCA. Is that transformational? In a time of scarce resources for the MCA, the question of ability to be transformational needs to be an increasingly important factor in compact choices, while avoiding merely pumping up the appearance of transformational impact in projecting results.
Lastly, accountability for results--impact in particular--is what’s truly innovative about the MCA and where the bulk of our energies should be focused. There is a lot riding on the ability of the MCA to deliver better results than existing aid programs. Its inability to do so could, in fact, jeopardize the already precarious support for development assistance. That doesn’t mean Congress should abdicate its important oversight role now, but it does mean a little patience and flexibility should be given to the MCA to implement so we can see what it does differently and whether it does indeed deliver better results.