The MCC: Will Simplicity Destroy Credibility?

The Millennium Challenge Corporation (MCC) Board of Directors faces a critical decision next month: it must determine whether to continue certain countries’ eligibility for grants for important development projects.  The MCC’s credibility is on the line.  Will it stick with its time-tested and successful procedures for selecting well-performing countries – all of its procedures, and not a simplistic and rigid misunderstanding of them?  Or will it miss the mark and tell deserving countries that were selected last year— and that have worked very hard over the past year based on that commitment— that they are now out simply because of misinterpretations of minor statistical fluctuations?  For the MCC’s sake and the sake of the people in the partner countries, let’s hope wisdom prevails and the board sticks with its principles and chooses the former.

The MCC was established in 2004 as a different kind of development agency.  The core idea was to select a subset of developing countries that had shown a strong commitment to good governance and sound economic and social policies, and then to work in close partnership with those countries to undertake investments with a high payoff for economic growth and poverty reduction. 

A key feature that distinguishes the MCC from other donors is its country selection process— it chooses countries with the right leadership, commitment, and track record to make the investments work.  In making its choices, the board takes into consideration a set of 20 quantitative indicators on governance and economic and social policy performance, along with other supplemental information on country readiness.  The starting point is whether a country surpasses the median score for its peer group on at least half the indicators, and in particular whether it passes the median score for control of corruption and at least one indicator of democratic rights. 

But—and this is where there often has been misunderstanding— the indicators are the starting point for selection, not the final determinant. The board also considers supplemental information “to address gaps, time lags, measurement error, or other weaknesses in the indicators to assist in assessing whether MCC funds might reduce poverty and promote economic growth in a country.”  It is worth focusing on the last phrase of this sentence, as it succinctly captures the ultimate goal of the process: assessing whether MCC funds might reduce poverty and promote economic growth in a country.  The founders of the MCC recognized both the limitations of the indicators and the importance of supplemental information in achieving that goal. 

I was part of the Treasury Department team in early 2002 that did the initial work on creating the indicators, and was deeply involved in the discussions with senior policymakers at that time about the role the indicators and other information would play in the selection process (I was also, until last year, the chief economist of USAID and through last September occasionally represented USAID on the MCC Board).  From those early meetings, the Bush administration understood both the strengths and the limitations of the indicators.  While they provide useful information, they are far from perfect and are at best proxies for actual performance.  Statistical errors, timing issues, imperfect surveys and other issues remind us that at the end of the day the indicators are just that— indicators, not hard facts. 

Toward that end, the MCC Board has used supplemental information several times to choose countries that didn’t quite meet the basic indicators test, as well as to not choose countries that did.  For example, it selected Mozambique and Georgia in 2004 for initial compacts, even though they fell slightly short on the indicators test.  Both of these choices resulted in excellent compacts.  Even more frequently, it has reselected countries that had initially passed the indicators and then slipped slightly.  It stuck with Senegal, the Philippines, Sri Lanka, and several other countries when their indicators dipped below the median.  In hindsight, it is clear that these decisions were wise for both the MCC and for the citizens of its partner countries, and that they succeeded in furthering the MCC’s basic goals and enhancing its credibility as a new kind of donor.

On the other hand, the board has often used supplementary information to not select countries that passed the indicators test, such as Egypt, Uganda, and Vietnam.  Moreover, it has also been more than willing to suspend or terminate compacts in countries where performance was seriously off-track, even if the country continued to pass the indicators, as it did in Nicaragua in 2008 and Malawi in 2012.  Time and again, the board has shown the wisdom of digging beyond the indicators to get a better sense of actual performance in making its decisions.

Let me be clear—I am not in any way suggesting that the indicators be thrown out, ignored, or overruled.  I am as strong an advocate for the indicators, properly understood, as anyone.  In particular, when initially selecting a country the board should put a strong weight on a country’s performance in meeting the indicators, and exceptions should be rare.  In most cases, countries that fall short for initial selection should be told to keep working at it.  Similarly, countries that achieve the basic benchmarks should be selected, unless there is compelling supplemental information to do otherwise.

But once a country is selected, the calculus changes for two reasons.  First, the MCC will quickly have at its fingertips much more accurate information about whether or not a country is ready to implement strong investments.  If the government shows indifference, acts like it is simply entitled to money or dramatically weakens its core polices, they are showing they are not ready.  By contrast, if the government leadership moves quickly to systematically analyze its greatest needs, undertakes sound initial work on possible investments, and opens a constructive dialogue with its citizens and partners, they are showing great promise.  Countries that have completed compacts or MCC Threshold programs provide even more evidence.  This kind of information on a real record of action is far more meaningful in meeting the objective of successful investments than the indicators can ever be on their own. 

Second, once a country is selected the MCC has begun to make a commitment to partnership.  In turn, as it begins to move forward the country makes a commitment to partner with the MCC.  That partnership— a two-sided commitment— should mean something. 

This brings us to the key decisions facing the MCC Board next month.  The new data on the indicators are out and four countries which the MCC selected last year have marginally slipped below the indicators benchmark: Benin, Morocco, Liberia and Sierra Leone.  The basic pattern in all four countries is the same.  The changes in the data are small and statistically insignificant, with no actual statistical evidence of any deterioration in policy; and meanwhile all four countries have shown clear evidence of the strong commitment and hard work necessary for successful investments.

Sierra Leone is a perfect example.  This year the point estimate for its “control of corruption” score is the 47th percentile, just below the median score needed for passing.  But both last year’s score and the median are well within the current margin of error (as with any data, there is always a margin of error).  More importantly, there is no statistically significant difference between last year’s score and this year’s score.  It is worth remembering that in statistics, movements of point estimates within the standard error have no statistical meaning.  In other words, even though the point estimate has declined slightly, there is zero— let me emphasize, zero—statistical basis to claim that there has been an actual increase in corruption in Sierra Leone.

Benin’s situation is nearly identical.  The point estimate for its control of corruption score slipped slightly below the median this year, but is not statistically significantly different from the median.  There is simply no evidence to suggest that Benin’s performance has actually declined.  

In Liberia (where I work as an advisor to the government), corruption is not the issue— Liberia comfortably passed the corruption indicator for the sixth year in a row (which should count for something).  Liberia got caught by a change in definition for the “natural resource protection” indicator, which it had easily passed the last several years.  A United Nations agency that creates one of the sub-indicators changed the definition of what counted as protected forest land in Liberia.  No one in the government knew about this definitional change and its impact on the indicators until the new data were published.  Meanwhile, this year Liberia moved ahead to designate additional forest lands for protection and has aggressively taken steps against illegal logging.  In other words, its actual policy performance has improved.  Should Liberia really be penalized by the MCC just because the definition of an indicator changed, even though no one told them about it, and while their actual policy performance improved?  What kind of partnership would that be?

But indicator scores aside, there is a far more important reason why the MCC should stick with these countries.  The MCC correctly selected these countries last year and in doing so initiated a two-way partnership.  The countries have undertaken a huge amount of work on their side of the equation.  In several cases, the work has been ongoing for several years: Benin and Morocco have completed MCC compacts, and Liberia a MCC Threshold program.  By all accounts, all of these efforts have been highly successful.  This past year, teams in Liberia and Sierra Leone have worked long hours to undertake substantial economic analyses to prepare for new investments and have held extensive discussions within their governments and with the public.  

These countries have done everything they have been asked to do over the last year, and have shown they can implement a compact that achieves the MCC’s ultimate goals of poverty reduction and economic growth.  They took the MCC’s pledge to honor the principal of partnership seriously.  In doing so they have provided more information to the MCC—information far more powerful than the indicators— that they are ready. 

One option for the MCC Board would be to take a narrow view of the selection process.  It could give supreme weight to the point estimates of the indicators, and ignore the proper interpretation of statistical variation, the supplemental information that actual performance has not declined, and all of the work these countries have undertaken during the past year.  But what would that achieve?  It would certainly not help the MCC’s credibility.  The ultimate test of the MCC’s credibility is not whether it sticks within the narrow confines of a statistical definition of passing an indicator—it is whether it selects countries that implement compacts that reduce poverty and promote growth.

It would not send a message to these countries that performance counts, or that the MCC is serious about upholding standards, since for these countries neither their actual performance nor their policy standards have declined.

Instead, deselecting these countries would send a deeply destructive and confusing message.  All of the hard work these countries did during the past year would be wasted.  The timeline for implementing beneficial projects would be delayed.  And the MCC’s reputation would take a big hit.  The MCC’s message to these countries last year was clear: if you work hard on a compact and if your policies do not deteriorate, we will continue to partner with you.  The new message would be equally clear: you should not have taken our message about partnership so seriously.  After all of your work this year based on last year’s selection, we have changed our mind even though there is no statistical evidence or other information to suggest that your performance has actually deteriorated.  You have done everything we have asked, but still you are out because that is what our rigid bureaucratic procedures say.

That would be a shame because it would signal that the MCC was beginning to lose its creativity and innovativeness, putting its own rules and bureaucratic procedures on a pedestal and starting to look like an old-fashioned donor with rigid conditionality and little understanding of true partnership with committed countries.  Its credibility with partners would suffer, big time.

Fortunately, the MCC Board is unlikely to make that choice.  I am sure the board will (and should) discuss these issues carefully.  But I am confident that in the end they will recognize that the MCC is about far more than a set of imperfect statistical indicators.  The MCC’s promise was to be a new kind of donor that would select counties that are committed to good policies and democratic governance, and then to work as true partners with those countries to fight poverty and promote growth.  Next month’s board meeting provides an opportunity for the board to reinforce the MCC’s core principles, refocus attention on its ultimate objectives, and help partner countries take a big step forward.  These countries want to stand with the MCC and the American people, and they have shown their commitment to doing so.  Let’s hope the MCC Board decides to stand with them.