In Memoriam — Just over a year ago, Steve Knack, a co-author of the paper on which this blog post is based, died. Steve was a towering figure in the political economy of development. His contributions were empirically rigorous and politically relevant—just like this paper. As one of my colleagues put it, "Even a year after his death, Steve continues to educate us." This blog is dedicated to his memory.
We are living in an era of resurgent nationalism and rising skepticism about the value of multilateral institutions. Prior to his appointment as the president of the World Bank, David Malpass told the U.S. Congress that the principal beneficiaries of the World Bank were “the people who fly in on a first-class ticket to give advice to governments.”
China’s new role as a financier of first resort for many low-income and middle-income countries has also raised fundamental questions about the raison d’être of multilateral development banks, such as the World Bank (see examples here, here, and here). Nobel Prize winner Michael Kremer and the Center for Global Development’s Michael Clemens make the case that the principal value of the World Bank does not lie in its lending activities for stand-alone public investment projects. Instead, they argue that the “Bank’s principal impact arises through its influence on national policies, and that economic theory suggests such influence is often best exerted multilaterally.”
Large-n empirical evidence suggests that the World Bank does indeed have outsized policy influence with public sector decisionmakers in the developing world. In 2014, AidData fielded a survey of nearly 7,000 development policymakers and practitioners in 126 low-income and middle-income countries that made it possible to benchmark the policy influence of bilateral and multilateral institutions. The results of the survey demonstrated that the World Bank has substantially greater influence over the direction, design, and implementation of government policies than most of its bilateral and multilateral peers. This finding was corroborated by a follow-up survey of development policymakers and practitioners that AidData conducted in 2017.
However, there is still relatively little evidence that speaks to the question of why—and—how the World Bank achieves such high levels of influence with governing authorities in the global south. Some have proposed that the World Bank is influential because of its use of development policy loans (DPLs), which do not support stand-alone projects but instead provide budget support in exchange for policy reform. A key feature of the DPL program is the fact that financial disbursements are linked to the adoption of specific policy conditions, so the World Bank’s ability to withhold disbursements from borrower governments could be the underlying source of its influence.
But the World Bank uses another instrument—that costs far less money—to influence government policy in its client countries: the provision of analytical and advisory services. Previous research suggests that these so-called “non-lending” services—for example, when the organization advises a government on the design of a conditional cash transfer program by analyzing the experiences of other countries that have implemented similar programs—are particularly influential when they are undertaken in collaboration with public officials from client countries, which may increase the resonance and perceived reliability of the World Bank’s analysis and advice.
However, a fundamental question remains unanswered: Are the World Bank’s lending or non-lending services more effective at influencing government policy? In a newly published working paper, we seek to answer this question. We do so by using AidData’s microlevel survey data on the policy influence of 150 aid agencies and development banks, as observed and experienced by their local counterparts in 126 low-income and middle-income countries. We focus on a specific subsample of survey respondents: 1,244 government officials from 121 countries who (a) participated in the 2014 Reform Efforts Survey, (b) reported having direct interactions with the World Bank, and (c) subsequently evaluated the World Bank’s influence on the direction, design, and implementation of government policy. Each of these survey respondents was asked to evaluate the World Bank’s level of policy influence over a specific time frame (during his or her tenure in a self-identified public sector position and institution) and in a specific country and policy area (his or her self-identified area of policy focus within a specific client country).
In order to measure exposure to the World Bank’s lending and non-lending services at an equally granular level (the country-policy cluster-year), we relied on data from the World Bank’s Business Intelligence Database and Development Policy Actions Database. If a respondent worked in a specific country and policy cluster—for example, Madagascar-Macroeconomic and Structural Policy, Guatemala-Human Development, or Sierra Leone-Governance—that received lending or non-lending services from the World Bank during a year in which he or she evaluated the policy influence of the World Bank, we identified the respondent as having experienced exposure to that particular type of World Bank “treatment.”
The fact that the survey asked respondents to evaluate the influence of the World Bank in a specific country, policy cluster, and time period allowed us to use statistical models with fixed effects at the level of country-policy cluster pairs. This approach is important because it made it possible to expunge the potentially confounding effects of observable and unobservable characteristics at the level of country-policy cluster pairs and isolate the variation within individual country-policy cluster pairs that is explained by the receipt of World Bank lending and non-lending services.
Our results provide strong evidence that the analytical and advisory products of the World Bank consistently influence the design, direction, and implementation of government policy in low-income and middle-income client countries. By contrast, we did not find strong evidence that the World Bank’s development policy lending—or investment lending—services consistently influence government policy.
As dyed-in-the-wool empiricists, we approached these preliminary findings with a healthy dose of skepticism and ran a battery of robustness checks to determine whether they might be spurious. Given that the World Bank often works with and through “sympathetic interlocutors” in client countries who share its policy preferences, one of our biggest concerns was that some of the public officials who answered these survey questions might have favored the World Bank’s policy ideas even in the absence of its analysis and advice. However, when we controlled for respondent willingness to engage in policy dialogue with the World Bank and respondent demand for reform analysis and advice from the World Bank, the effect of the organization’s non-lending services remained unchanged. Nor did it change when we used a statistical model that explicitly accounted for self-selection bias. We also ran placebo tests and other sensitivity checks to gauge the robustness of the results. All of our empirical results pointed in the same direction: The World Bank enjoys outsized policy influence primarily because of its ideas (delivered through analysis and advice) and not because of its money (delivered through loans).
To put these findings in perspective, the World Bank spends about $200 million on non-lending services and $30 billion on lending services every year, which means that for each dollar that the World Bank spends on non-lending services, it spends approximately $150 on lending services. Therefore, if a central goal of the World Bank is to influence government policy in low-income and middle-income countries, our results imply that it can achieve better value for money by doubling down on the provision of analysis and advice.
Our findings also have implications for other international development organizations. As low-income and middle-income countries reduce their reliance on official development assistance and assume greater responsibility for direct service delivery activities, many aid agencies and development banks are refocusing their efforts on helping partner countries reform laws, institutions, regulations, and customary practices in ways that they hope will promote economic development and effective governance in the long run. This reallocation of time, money, and effort may be appropriate, but there is a growing gap between the stated objectives of international development organizations and the metrics of success that they use to judge their own performance. Easily observable output and outcome measures—such as the amount of economic output generated, the number of children vaccinated, and the number of kilometers of roads constructed—are useful for measuring the success of direct service delivery activities. But most aid agencies and development banks lack credible ways of measuring their influence on the policy priorities of partner countries. Our study demonstrates that relatively inexpensive investments in the collection of data can help international development organizations that aspire to influence government policy measure the effectiveness of the instruments that they create for this purpose.