MCC Has a Corruption Problem

At the heart of the Millennium Challenge Corporation (MCC) is a focus on good governance. The agency is known—and broadly liked—for its selectivity, choosing only relatively well-governed countries for its large five-year, growth-focused grants (or “compacts”). The idea behind the focus on good governance was to reward countries taking responsibility for their own development, create incentives for reform, and potentially increase the effectiveness of MCC investments. Of all the governance criteria MCC assesses, none is as singularly important as corruption, which, historically, has weeded out more countries for eligibility than any other individual factor. It is, however, difficult to measure with precision, which can (and has) lead to poor decisions when interpreted too rigidly, resulting in cutting off, purely on the basis of indicator rules, compact partnerships with countries that have had no demonstrable change in their anticorruption environment. If you care about corruption, this isn’t the way to go about emphasizing that.

While MCC has long been aware of this problem, it is finally poised to formally address it. As MCC flagged for its board of directors in June, the agency is considering options for adjusting its approach to the corruption criteria. Though any change would be small, it would, if done well, eliminate the periodic need to trade off “playing by the rules” and the imperative to use data wisely and responsibly. The question that remains is how to craft a guideline that remains rule-based but provides meaningful assessment of country performance, is free of perverse incentives, and allows MCC to maintain a strong stance on corruption. Below I outline some options—including what I see as the best choice—and address reservations that such a change would fundamentally alter MCC’s focus on corruption or hamper its power to incentivize anticorruption reforms.

The problem with the current measure of corruption

For background on MCC’s selection process, see MCC’s official document or my short synopsis (see section “How the Selection Process Works,” p. 2-4).

There’s a lot to like about MCC’s evidence-based selection system, including its transparency and the way it helps focus eligibility decisions on criteria related to policy performance rather than geopolitical considerations. Indeed, the board—which is responsible for all eligibility decisions—has an important role to play in ensuring MCC does not invest its scarce funds in the most corrupt countries or in countries where corruption is on the rise. The scorecard—including the Control of Corruption indicator—can help it do that. But because the data imperfectly reflect both ranked performance and changes over time, the indicator offers only one part of the story. Unfortunately, the hard hurdle on Control of Corruption—a rule that countries must pass this indicator to meet the overall scorecard criteria for eligibility—while not meant to be an ironclad rule, sometimes encourages more rigid interpretation of the data than the indicator can reasonably support. This problem takes two main forms.

The first is that MCC’s process of ranking countries assigns greater differences among countries than what is actually found in the data. The Control of Corruption indicator comes with a calculable margin of error that shows the range of possible values a country’s score could take. The point estimate—the score MCC uses—is the midpoint of this range and what MCC uses to calculate each country’s distinct percentile rank. Countries with scores above the median (50^th percentile) of the income-level peer group pass; those below fail. However, if you take into account the margin of error, as shown in the figure below, countries from the 32^nd percentile(Cambodia) to the 77^th percentile (Djibouti) have overlapping—that is, statistically indistinguishable—scores, yet only those including and to the right of Mozambique pass. That is to say, around half of all MCC candidate countries have Control of Corruption scores that are statistically indistinguishable from the median and from each other, yet roughly half of these roughly equivalent countries could be considered for MCC partnership while the other half is deemed too corrupt.

Image

 
The second problem is that the Control of Corruption indicator, as currently employed by MCC, is not well-suited for monitoring performance on a year-to-year basis. Clear upward or downward trends do sometimes emerge over several years, but small year-to-year changes in a country’s Control of Corruption score are often just statistically insignificant “noise,” unlinked to real deterioration (or improvement) in policy. Unfortunately, however, these small, otherwise unimportant changes can make a passing-to-failing difference for countries in the large cohort whose scores cluster around the median. This is particularly problematic when it happens to a country in the middle of developing a compact since it generally must be reselected each year during program development (typically 2-4 years) until the compact is signed.

Another reason countries can move from passing to failing irrespective of policy deterioration relates to changes in the composition of the comparator group as countries shift among income categories. In FY2017, for instance, some high-performing countries joined the lower-middle-income group and some lower performers moved out, pushing the median score slightly up. A small increase like this can push those previously just above the line into failing territory—as it did to Mongolia. While neither of these circumstances—minor score fluctuations or changes in the median—suggests the country is suddenly more corrupt than it was when it was initially given the stamp of approval, a rigid application of MCC’s current scorecard rules suggests the board should consider cutting these countries off.

Dissatisfaction with MCC’s use of the Control of Corruption hurdle has led some of my colleagues to advocate dropping it completely. While their justification is sound, I would contend that the existing rules are acceptable for the initial selection decision (i.e., the first time a country is picked to begin developing a compact), even in the light of the data limitations described above. It’s true that being in or out comes down to little more than luck for many middle-performing countries, but it’s at least a transparent basis for eligibility decisions. For reselection decisions, on the other hand, I’ve long argued (here p. 4-8, here, here), the current rules are deeply inappropriate. Once a country is selected, it should only be cut off on the basis of an actual, identifiable pattern of actions that demonstrates real policy decline. MCC should not cut off a country (by not reselecting it) or threaten to cut it off (by requiring it to pass the scorecard before proceeding to compact signing) simply on the basis of the movements of imprecise data.

Unfortunately, MCC’s board hasn’t always seen it that way, deciding not to reselect Benin, Sierra Leone (FY2014), and Kosovo (FY2017) for compact eligibility when they narrowly failed the Control of Corruption indicator. MCC even admitted that Benin and Sierra Leone had exhibited no deterioration in policy. In the end, Sierra Leone and Kosovo were relegated to MCC’s much smaller threshold program. Benin was ultimately reselected for compact eligibility the following year, but only after compact development was slowed by the earlier decision. In all of these recent cases, the board evidently weighed the need to be seen playing by the rules more heavily than the need to account for data limitations. This is why I’m heartened that MCC is exploring small but absolutely critical options to adjust its approach to the Control of Corruption indicator for reselection decisions.

How to fix the problem

There are a number of different forms such an approach could take, but my proposal for a clear best choice is as follows:

For a country being considered for reselection, it should be considered to “pass” the Control of Corruption indicator if its current year’s score is within the margin of error from its score the year it was selected.

Simply put, if the country’s score is statistically no different from the year in which it was determined to perform sufficiently well, its pass/fail performance on the indicator should be interpreted in a similarly consistent manner. In fact, the indicator’s creators specifically mention that the margin of error is included to “enable users to avoid over‐interpreting small differences between countries and over time in the indicators that are unlikely to be statistically—or practically—significant.” This rule would better align data rules with smart data interpretation, eliminating the need for the board to make tradeoffs between them.

Such a rule would not guarantee countries continued eligibility, even if their score is within the margin of error of the score the year it was selected. As with all selection decisions, the board has discretion to determine whether a country that passes the indicator criteria should be selected as eligible. A wide range of supplemental information on the actual policy environment informs this decision, and if the board determines a real policy decline is occurring, then the partnership should be reconsidered. In the absence of a concrete concern, however, it should continue if the score is statistically unchanged.

Alternative (less promising… or even damaging) solutions

There are other options MCC may be considering, some of which would entail the formalization of various ad hoc pronouncements the agency has made before in explaining individual eligibility decisions. All of these have more significant drawbacks. They are either unconducive to the creation of a simple “rule,” offer little improvement over the status quo, or, in some cases, create perverse incentives.

Using a different, more precise indicator: MCC has long searched for an alternative corruption indicator that is more precise and less noisy. But to date there are no plausible alternatives that don’t have at least equivalent methodological constraints and that meet MCC’s other criteria of public availability, broad country coverage, and regular periodicity.

Using “trends” in score as the basis for a rule: Indicator trajectory was one of the factors that MCC cited in its decision to discontinue compact eligibility for Kosovo last year. Looking at trends in a country’s Control of Corruption score over time can be instructive and should be part of the board’s review. When clear improvements or declines in indicator score appear over the span of several years, they are often associated with real shifts on the ground. See, for instance, the trajectories of Cote d’Ivoire and Mozambique in the figures below. The first is a country making marked progress toward stable governance post after a long turbulent period while the second has weathered major scandals that have prompted donors to pull funds in recent years.

Image

 
But not all trends are as clear cut as the examples above, raising the question of what degree (and consistency) of downward slope is sufficient to trigger concern? A country whose precipitously declining trend line looks like Mozambique’s? Probably. One whose downward trajectory is flatter overall and peppered with ups and downs, like Liberia’s? It’s harder to call this a real decline.

Image

 
Furthermore, noisy scores that bounce around from year to year can create similarly rapidly shifting trends. Take Sierra Leone, for instance. From 2010 to 2014, its trend was negative. But shift forward just one year and the trend becomes positive.

Image

 
Applying a “three strikes and you’re out” rule: Under this rule, a country could fail the Control of Corruption indicator twice with a free pass (as long as there was no significant deterioration in actual anticorruption policy), but the third consecutive failure would be considered a “real” failure with the potential consequence of non-reselection. This would be a poor rule choice for two reasons. First, it still allows for countries to be cut off for reasons other than policy performance, if their decline in score does not reflect an actual decline in policy. Second, it is full of perverse incentives. For a country that has failed twice, MCC would have every incentive to rush the compact to the finish line before the country had a chance to fail a third time; this could have negative implications for program readiness and quality.

Accepting failure during compact development but requiring the country to pass to proceed to compact signing: MCC has made this kind of proclamation before, and luckily, both times things worked out. But that’s all it was—luck that a high stakes gamble on the behavior of imprecise data yielded a convenient outcome at the right time. The problems with this option mirror those of the option above. It is a waste of US and partner country time and resources to fully develop a program only to have it pulled at the end purely on the basis of noisy data. It also creates perverse incentives for MCC to rush programs to completion in a year a country passes just in case it doesn’t pass the next year.

Addressing reservations

The major challenge with any change to how MCC uses the Control of Corruption indicator is one of optics. The agency’s selection system is admired in part because of the way it strongly signals the importance of fighting corruption. Supporters like the idea that it keeps MCC from channeling funds to the most corrupt countries and incentivizes reform in countries seeking to gain MCC eligibility. Not only that, Congress grants MCC substantial freedom from earmarks and other spending directives because it recognizes that the agency’s rule- and evidence-based systems serve as an alternative form of accountability for sound decision making. Given these interests, it’s important to address potential concerns surrounding the change:

• Would this new approach be used so much that it fundamentally alters how MCC assesses corruption performance for countries? 
  No. The new rule would come into play very infrequently. In its entire 14-year history, MCC has faced decisions about whether to reselect countries that failed the Control of Corruption indicator during compact development only 15 times (out of a total of 101 individual reselection decisions). This suggests the issue would come up for one country per year, on average. And in many years it will be a complete non-issue. For six of MCC’s 14 years, no countries up for reselection failed the Control of Corruption indicator. The current rules would prevail for most decisions. The new approach would merely be a small way to hedge against the rare cases in which an action based on the current rules could result in a poor decision.  
• Does it soften MCC’s stance on corruption? 
  Not at all. The board maintains every right to choose not to reselect a country that exhibits an actual deterioration in its anticorruption environment. In fact, by shifting the focus from indicator score (as long as it is within the margin of error of its score at initial selection) to actual policy environment, it makes MCC much more serious about corruption because it requires the agency to point to a concrete set of concerns that a country could then presumably address. On the other hand, a rigid interpretation of the current rules can (and has) put MCC in the awkward and wishy-washy position of having to say to a country, “no, it’s nothing in particular that you’ve done that caused you to fail, but a rule is a rule… Maybe try to address a few of these problematic things, even though we have no idea whether doing so will affect your Control of Corruption score because it’s just not that sensitive or precise. Sorry!” Which of these options offers a more serious stance on corruption? (Hint: it’s not the latter.)  
• Does it look bad for MCC to partner with countries whose Control of Corruption scores are red on the scorecard? 
  This is admittedly tricky from an optics perspective. However, being more explicit about the rules of interpretation would go a long way to help. Taking it further, MCC could—and arguably should—match the scorecard color to performance according to the rules. That is, if a country passes MCC’s indicator rules—including the minor change in approach to interpreting corruption for countries up for reselection—the scorecard should reflect that passage in green. This would not be the first time special, non-median-based rules are applied to a particular set of countries. MCC changed the passing threshold for the Immunization Rates indicator for lower-middle-income countries to a fixed threshold rather than the median because using the median as a cutoff was yielding illogical decisions. This adjustment could—and should—be viewed similarly.
• Does it compromise the MCC scorecard’s purported ability to incentivize reform (a.k.a., “the MCC Effect”)? 
  It’s unlikely. There are some real, concrete examples of MCC’s selection system contributing to reform conversations, but it likely works idiosyncratically rather than systematically. It would be a mistake, therefore, to let broad, unproven assumptions about how the “MCC Effect” works prevent the agency from addressing specific, known problems with the way its data is sometimes used. Furthermore, it’s far from clear that the incentive effect MCC scorecards have in the area of corruption is particularly meaningful. Research (and an MCC-specific example) show that external assessments like MCC’s scorecards may play a role in shaping anticorruption policies, but likely in a way that responds more to donor demands rather than in a way that actually combats corruption.