Sections

Commentary

Why is it so hard to use privately collected big data to improve economic measurement?

Hutchins Center event - Can big data improve economic measurement

In an era of big data, where our every move is seemingly captured and recorded, traditional means of collecting data for official measures of the economy – from conducting surveys to sending examiners to check prices on store shelves – appear increasingly outdated. Privately collected big data from corporations like Visa, JPMorgan Chase, supermarket scanners and health insurance companies can be an incredibly rich source of information on the income and spending of households, the revenues and expenditures of business, and the prices and quality of products. However, difficulties related to interpretation and access remain significant barriers to properly utilizing such data.

As part of its Productivity Measurement Initiative, the Hutchins Center on Fiscal & Monetary Policy at Brookings hosted a panel of experts in March 2019 to discuss the potential of, as well as the obstacles to, using big data in economic measurement. The panel featured Michael Brown, principal U.S. economist for Visa; Fiona Greig, managing director of the JPMorgan Chase Institute; Erica Groshen, former commissioner of the Bureau of Labor Statistics (BLS); and Crystal Konny, consumer prices branch chief at the BLS. Here’s some of what they had to say.

When statistical agencies get access to big data, what issues arise with interpreting it?

As businesses change, so do the data they collect, which can make teasing out the underlying economic information challenging. For instance, if Visa adds many new business customers from one month to the next, purchases using Visa cards will go up, but that increase isn’t necessarily a reflection of an increase in overall consumer spending. As Brown and Greig highlighted, these changes can be hard for even the company’s own economists to keep track of, let alone a statistical agency without inside knowledge of what’s changing and why.

There are additional challenges for statistical agencies, who have no control over what data gets collected or how the data might change over time. Konny pointed out that privately collected data often doesn’t contain the right information or the level of detail statistical agencies need.

A company also might choose to change the structure of data or how that data is stored. This makes it difficult for statistical agencies to rely on privately collected data to produce data series that need to be both timely and accurate. Konny described how disruptive changes to privately collected big data can be based on the BLS’s experience working with data from a major retailer.

What are the obstacles to statistical agencies embracing privately collected big data?

Privacy and trust are a big issue, and this makes sharing corporate data difficult. Brown said Visa has two major concerns – one is privacy and trust, and the other is violating financial securities laws by providing what might be deemed insider information on Visa’s earnings.

It is not only hard, and potentially expensive, for statistical agencies to gain access to privately collected data—incorporating the data in the production of economic statistics can also be challenging. Groshen highlighted that, because statistical agencies collect data on a voluntary basis, there is always a risk that providers will stop handing over their data if they no longer feel that it is worthwhile.

Groshen also pointed out that integrating the new data into existing methods is time consuming, and agencies face a tradeoff between allocating stretched budgets towards experimenting with new data and producing vital economic statistics.

What are statistical agencies doing?

Despite these issues, statistical agencies are working on ways to use big data to supplement or improve their economic measures. Crystal Konny gave one example of how the BLS consumer price index team is experimenting with privately sourced big data to improve the measurement of healthcare prices, an area where the survey response rate is particularly low.

Groshen said that we should nevertheless expect statistical agencies to continue to be slow and cautious about using new data and new methods, and that’s what we should want.

The Hutchins Center conference drew from papers presented at the 2019 Conference on Income and Wealth (CRIW), which, among things, explored the use of big data sources by statistical agencies to improve their current measures.

Authors