Real Clear Markets

Why Is the Price of Oil So Hard to Predict?

At a recent annual retreat of elite Wall Street money managers, the hosts reported on how well last year's attendees predicted various economic measures. Forecasting is notoriously difficult, and nobody guessed how high stock prices would be today. But the most interesting miss was that the group was way too low on oil prices. Should we be surprised?

Oil prices are hard to forecast because they are highly sensitive to shocks in both global demand and supply. Forty years ago, historic disruptions to global oil supplies destabilized the world economy for a decade. When the oil producing countries of the Middle East nationalized their oil industries and formed the OPEC cartel that quadrupled world oil prices, they generated the steep recessions of the mid-1970s. And when Iran's oil supplies were disrupted by the overthrow of the Shah of Iran by Muslim clerics in 1979, a new surge in oil prices led to a second round of even deeper recessions.

This crisis was followed by a movement to greater energy efficiency and two decades of relative calm in oil markets. The price of oil fluctuated between $15 and $35 a barrel, which represented a substantial decline in the real price of oil.

That relative calm changed over the past ten years, with both the demand and supply sides of the market changing in big ways. On the demand side, the rapid development of China, India and Brazil quickened world GDP growth and oil demand, pushing oil prices from $25 to $125 a barrel between 2001 and early 2008. The Great Recession brought oil prices below $50 in 2009. But despite only tepid recoveries in the advanced economies, oil prices recovered promptly and have hovered near $100 since 2011.

On the supply side, the development of fracking as a new technology for getting oil and gas out of oil shale has reversed a long decline in North American oil production. Shale oil has been hailed as a move to energy independence for the region, a source of good jobs for US workers, and a major step in reducing the US trade deficit. Perhaps more importantly, it is also a dominant factor in the growth of world supplies and the world price of oil. The development of shale oil has already raised North American production by nearly 5 million barrels a day, which is over 5 percent of today's total global demand. Had shale oil not come along, oil prices would be much higher today.

For the longer run, the future of oil prices will depend mainly on the race between growing demand in the emerging economies and growing supplies from shale oil deposits around the world. Political change, like the recent decision to replace the state oil monopoly in Mexico with private industry, may also make some difference. As for how far prices might move, on the downside, there is some support to prices from the relatively high cost of producing shale oil. The upside for prices is more open ended, although very high prices would encourage the development of infrastructure to make natural gas a useful automobile fuel.

Over the short run, departures from this uncertain long-run trend are vulnerable to political changes in unstable parts of the world. Iran's oil production is down by a quarter, presumably because of the embargo on shipments into the world market. If the new government in Teheran improves relations with the US, the lifting of the embargo would restore that production. On the other hand, in Iraq, where oil production has been expanding, experts fear that a civil war could erupt.

When the guesses from this summer's money managers retreat are revealed next year, we should not be surprised if the oil price was again the toughest prediction to make.