When There’s No Such Thing as Too Much Information

INFORMATION overload is a headache for individuals and a huge challenge for businesses. Companies are swimming, if not drowning, in wave after wave of data — from increasingly sophisticated computer tracking of shipments, sales, suppliers and customers, as well as e-mail, Web traffic and social-network comments. These Internet-era technologies, by one estimate, are doubling the quantity of business data every 1.2 years.

Yet the data explosion is also an enormous opportunity. In a modern economy, information should be the prime asset — the raw material of new products and services, smarter decisions, competitive advantage for companies, and greater growth and productivity.

Is there any real evidence of a “data payoff” across the corporate world? It has taken a while, but new research led by Erik Brynjolfsson, an economist at the Sloan School of Management at the Massachusetts Institute of Technology, suggests that the beginnings are now visible.

Mr. Brynjolfsson and his colleagues, Lorin Hitt, a professor at the Wharton School of the University of Pennsylvania, and Heekyung Kim, a graduate student at M.I.T., studied 179 large companies. Those that adopted “data-driven decision making” achieved productivity that was 5 to 6 percent higher than could be explained by other factors, including how much the companies invested in technology, the researchers said.

In the study, based on a survey and follow-up interviews, data-driven decision making was defined not only by collecting data, but also by how it is used — or not — in making crucial decisions, like whether to create a new product or service. The central distinction, according to Mr. Brynjolfsson, is between decisions based mainly on “data and analysis” and on the traditional management arts of “experience and intuition.”

A 5 percent increase in output and productivity, he says, is significant enough to separate winners from losers in most industries.

The companies that are guided by data analysis, Mr. Brynjolfsson says, are “harbingers of a trend in how managers make decisions.”

“And it has huge implications for competitiveness and growth,” he adds.

The research is not yet published, but it was presented at an academic conference this month. The conclusion that companies that rely heavily on data analysis are likely to outperform others is not new. Notably, Thomas H. Davenport, a professor of information technology and management at Babson College, has made that point, and his most recent book, with Jeanne G. Harris and Robert Morison, is “Analytics at Work: Smarter Decisions, Better Results” (Harvard Business Press, 2010).

And companies like Google, whose search and advertising business is based on exploiting and organizing online information, are testimony to the power of intelligent data sifting.

But the new research appears to be broader and to apply economic measurement to the impact of data-led decision making in a way not done before.

“To the best of our knowledge,” Mr. Brynjolfsson says, “this is the first quantitative evidence of the anecdotes we’re been hearing about.”

Mr. Brynjolfsson emphasizes that the spread of such decision making is just getting started, even though the data surge began at least a decade ago. That pattern is familiar in history. The productivity payoff from a new technology comes only when people adopt new management skills and new ways of working.

The electric motor, for example, was introduced in the early 1880s. But that technology did not generate discernible productivity gains until the 1920s. It took that long for the use of motors to spread, and for businesses to reorganize work around the mass-production assembly line, the efficiency breakthrough of its day.

The story was much the same with computers. By 1987, the personal computer revolution was more than a decade old, when Robert M. Solow, an economist and Nobel laureate, dryly observed, “You can see the computer age everywhere but in the productivity statistics.”

It was not until 1995 that productivity in the American economy really started to pick up. The Internet married computing to low-cost communications, opening the door to automating all kinds of commercial transactions. The gains continued through 2004, well after the dot-com bubble burst and investment in technology plummeted.

The technology absorption lag accounts for the delayed productivity benefits, observes Robert J. Gordon, an economist at Northwestern University.

“It’s never pure technology that makes the difference,” Mr. Gordon says. “It’s reorganizing things — how work is done. And technology does allow new forms of organization.”

Since 2004, productivity has slowed again. Historically, Mr. Gordon notes, productivity wanes when innovation based on fundamental new technologies runs out. The steam engine and railroads fueled the first industrial revolution, he says; the second was powered by electricity and the internal combustion engine. The Internet, according to Mr. Gordon, qualifies as the third industrial revolution — but one that will prove far more short-lived than the previous two.

“I think we’re seeing hints that we’re running through inventions of the Internet revolution,” he says.

 

STILL, the software industry is making a big bet that the data-driven decision making described in Mr. Brynjolfsson’s research is the wave of the future. The drive to help companies find meaningful patterns in the data that engulfs them has created a fast-growing industry in what is known as “business intelligence” or “analytics” software and services. Major technology companies — I.B.M., Oracle, SAP and Microsoft — have collectively spent more than $25 billion buying up specialist companies in the field.

I.B.M. alone says it has spent $14 billion on 25 companies that focus on data analytics. That business now employs 8,000 consultants and 200 mathematicians. I.B.M. said last week that it expected its analytics business to grow to $16 billion by 2015.

“The biggest change facing corporations is the explosion of data,” says David Grossman, a technology analyst at Stifel Nicolaus. “The best business is in helping customers analyze and manage all that data.”