Kemp: Data Availability Bias in the Oil Market

Accurate information is essential for good decision making, so it is remarkable how little reliable and timely data exists about production and consumption of oil outside the US.
Accurate information is essential for good decision making, so it is remarkable how little reliable and timely data exists about production and consumption of oil outside the US.

John Kemp is a Reuters market analyst. The views expressed are his own


LONDON, July 7 (Reuters) – Why is there such good data about oil in the United States but such poor data about everywhere else?

Accurate information is essential for good decision-making, so it is remarkable how little reliable and timely data exists about the production and consumption of crude oil and refined fuels outside the United States.

The situation in the other advanced economies, not to mention emerging markets, is mostly guesswork.

The result is that oil analysts cannot even agree on production and consumption yesterday and today, let alone predict what will happen tomorrow.

And because the best and most readily accessible data is for the United States, the market puts excessive emphasis on what happens there and neglects developments elsewhere.

The obsession with weekly rig counts, production estimates and crude inventories in the United States as a sign of wider supply-demand trends in the oil market has been a case in point.

But as long as U.S. data is more accurate, detailed and timely than the numbers for other countries, this example of "availability bias" is set to continue.

Energy Crisis

Some U.S. data comes from private companies such as Baker Hughes, which inherited the decades-old rig count from the Hughes Tool Company, but most is produced by the federal government.

The U.S. Energy Information Administration, the independent statistical and analysis arm of the Department of Energy, provides by far the best data on oil and other energy markets anywhere in the world.

The EIA's magnificent data collection and publication effort was born amid bitter recriminations about the federal government's failure to foresee the energy shortages of the 1970s.

In early 1974, just months after Arab countries embargoed oil exports to the United States, more U.S. voters blamed oil companies and their own government for shortages and price rises than the oil exporters.

"The real trouble in the energy crisis is that the government does not seem to know how serious the shortage will be", according to 53 percent of those questioned in one survey conducted in February 1974.

Widespread suspicions arose that oil and gas companies were exacerbating the crisis by deliberately withholding production to drive prices higher.

Even as late as 1979, more people were blaming high prices on companies deliberately withholding oil and gas production (65 percent) and refusing to drill wells unless prices were raised (61 percent) than OPEC (59 percent).

"Neither industry nor government had given the public advanced warning of energy shortages. Even after they occurred, there seemed to be no clear explanations or comprehensible data," historian Richard Vietor wrote later about the confusion and conspiracy theories ("Energy Policy in America", 1984).

Reputable publications including Newsweek, Time and the Washington Post were filled with articles suggesting domestic oil and gas producers were colluding with the Organization of the Petroleum Exporting Countries to engineer shortages and raise prices.

"The news media only added to the confusion. Mistrust abounded as to the reality of the crisis and its causes," Vietor concluded.

Conspiracy theories flourished because it turned out the government relied almost exclusively on the oil and gas industries for information on the state of production and reserves.

"In terms of getting the facts in the energy area, the federal government has completely delegated their responsibility to the oil industry," complained the chairman of a congressional committee. "This deplorable situation has to end."

Senior officials from the administration testified they were satisfied with data they received from the National Petroleum Council, the American Petroleum Institute, the American Gas Association and other industry associations, but Congress was adamant the government needed its own independent information.

"The American people want to know if there is an oil shortage. The American people want to know if there are oil tankers anchored offshore waiting for a price increase or available storage before they unload. The American people want to know whether major oil companies are sitting on shut-in wells and hoarding production in hidden tanks and at abandoned service stations," Senator Henry "Scoop" Jackson demanded to know.

Jackson chaired the powerful Senate Permanent Subcommittee on Investigations and turned the heat up on the industry.

"The American people want to know why oil companies are making soaring profits. The American people want to know if this so-called energy crisis is only a pretext, a cover ... to raise prices, to repeal environmental laws and to force the adoption of new tax subsidies," Jackson wondered in January 1974.

Data Demand

Between 1973 and 1975, at least 25 of the 39 permanent congressional committees then in existence held hearings into some aspect of the energy crisis.

The recurrent theme through the 1970s was the need for better data. Congress published reports on everything from "Conflicting information on fuel shortages" (1974) and "Energy data requirements of the federal government" (1974) to "Energy information shortcomings and the gasoline shortage" (1979).

From this was born the most ambitious and comprehensive effort ever undertaken by any country in times of peace to collect detailed information on the production, processing, distribution and consumption of energy.


View Full Article


Generated by readers, the comments included herein do not reflect the views and opinions of Rigzone. All comments are subject to editorial review. Off-topic, inappropriate or insulting comments will be removed.