The two blind spots of megaproject appraisal

A database of 500 major projects developed over the past 25 years in 100 countries documents how difficult it is to deliver major projects worldwide. Data show that eight in ten of the projects reviewed performed below expectations in economic terms, approximately two in three ran over budget (with a median cost overrun of 47%), and more than half finished late (with a median delay of 61%). 

These findings are not new. Several academics and practitioners have documented similar patterns over the past decades, and many reports have sought to explain why. Governments and development institutions have invested considerable effort in finding alternative ways of appraising and delivering megaprojects to avoid these problems, including better appraisal manuals, stronger technical safeguards, and more sophisticated risk analysis and risk frameworks. Despite advances in the field, the pattern persists, and the figures above suggest the underlying problem remains unsolved.  

To change these results, a different way of understanding the problem is needed. It seems unreasonable to assume that these previous problems persist because decisionmakers lack the necessary knowledge. This commentary examines two changes that many practitioners recognize intuitively but that have not been systematically incorporated into practice. 

First, a different role should be assigned to politics when appraising megaprojects. Appraisal techniques and methodologies have been built through the lenses of economics and project management. Under this view, when a project is analyzed, politics is usually considered a constraint. It is not treated as a variable that a technocrat analyzing a project can influence. Technocrats are used to developing geological studies to understand ground conditions and adapt the engineering design, or conducting a stakeholder analysis to develop a plan and avoid protests. Environmental assessments are produced to check for affected flora and fauna, so that teams act accordingly. Different estimates of the exchange rate are generated to work on a scenario analysis and check how this variable may affect a specific project. The norm is to treat these kinds of technical, social, environmental, and economic variables as such, but not politics, which has been understood as a constraint that has to be accepted. The first proposed change is to adopt a more public policy-based perspective and consider political conditions as variables that can be measured and incorporated into project analysis.   

Second, a different way of understanding and estimating project risks is needed. Risks are dynamic and interdependent, but they are normally analyzed statically and separately. There have been some academic efforts in this direction, but they have not reached mainstream practice. Any methodology involves a trade-off between detail and completeness on one hand and ease of use on the other, and academic work in this area has not yet been translated into methods simple enough for routine use. This helps explain why the static risk matrix remains standard, but experience in practice suggests that a more dynamic analysis is needed. Megaproject risks evolve, migrate, and propagate through a complex system of relationships among technical, social, environmental, economic, and, of course, political variables that are embedded in the risk profile of the analyzed project. 

This commentary refers to these two issues as the two blind spots of megaproject appraisal practice. The predominant methodologies used and proposed by international and multilateral organizations tend to reproduce them. In seminars and workshops on this topic organized by these organizations, recommendations typically focus on improving pre-investment studies. Better studies would help, but on their own, they are not enough, because whether such studies are completed before a project advances depends heavily on political conditions. 

Insisting on better pre-investment studies when an election is near and a decisionmaker needs to show progress is very different from suggesting them when there is a political window to do so. The typical technocratic solution to this issue has been to institutionalize an independent review of project appraisals to screen out poorly justified projects. However, such rules can be circumvented, and a motivated decisionmaker can often find a way around them. And is this wrong? Perhaps it is not. Politicians may be responding to mandates, coalitions, and windows of opportunity that result from democratic processes, and technocratic analysis should account for these dynamics rather than exclude them. 

Furthermore, the same international institutions often rely on static risk analysis for the reasons explained above. A static risk matrix is simple to construct and easy to understand; it estimates qualitatively or quantitatively the probability of occurrence and potential impact. In fact, multilateral banks use the same tool to analyze their interventions, such as project loans. 

There is room to rethink the way that megaproject risks are analyzed in countries and in international development organizations. To that end, a theoretical system dynamics model of risks was constructed, related to technical, environmental, social, economic, and political conditions. Relationships among the variables included were modeled and calibrated using the database of 500 megaprojects. For each of those projects, 65 variables were coded, including, for example, political urgency, public salience, cross-party support, and public-value narrative quality, among others. Each variable was defined with a codebook and scored using predefined thresholds, based on how strongly the condition is present in the evidence.     

Several patterns emerged from the analysis. One particularly clear example puts numbers on something many practitioners intuitively understand, which is the timing of public announcement relative to feasibility work. When projects were announced before pre-investment studies were completed, the database shows a median of 37% cost overruns and 49% delays. When the project was announced after the pre-investment studies, the typical cost overrun was zero, with a 5% average delay. However, this variable is rarely included in risk matrices used by governments or international organizations when studying a megaproject.  

The broader lesson from the database is that political and dynamic risks are most useful to assess early, when decisions are still reversible. The same model can also be applied to projects still in preparation through an AI-assisted diagnostic tool now being tested by specialists and practitioners. Used this way, the model can show how current risks could migrate, surface lessons learned from similar projects around the world, and suggest mitigation strategies, both technical and political. It does not produce point estimates of risk; the aim is to support more informed discussion of the project among the public- and private-sector participants involved. 

These two blind spots can be incorporated more explicitly into decisionmaking. The evidence points to a need to change how megaproject risk is assessed, and the database now makes it possible to compare the political conditions of current projects with those of similar projects worldwide. System dynamics modeling can also show how risks are likely to migrate and propagate under a given strategy, and how they change when one or more conditions are adjusted. AI-assisted tools may help to make more sophisticated appraisal practical, bringing political context and evolving risks into everyday project decisions.