Using Simulated Experience to Make Sense of Big Data

Book description

In an increasingly complex economic and social environment, access to vast amounts of data and information can help organizations and governments make better policies, predictions and decisions. Indeed, more and more decision makers rely on statistical findings and data-based decision models when tackling problems and forming strategies. So far, discussions of data-based decision making have centered mainly on analysis: data collection, technological infrastructures and statistical methods. Yet another vital issue receives far less scrutiny: how analytical results are communicated to decision makers. Data science, like medical diagnostics or scientific research, lies in the hands of expert analysts who must explain their findings to executive decision makers who are often less knowledgeable about formal, statistical reasoning. Yet many behavioral experiments have shown that when the same statistical information is conveyed in different ways, people make drastically different decisions. Description, the authors note, is the default mode of presenting statistical information. This typically involves a verbal statement or a written report, which might feature one or more tables summarizing the findings. But the authors’ own research suggests that descriptions can mislead even the most knowledgeable decision makers. In a recent experiment, they asked 257 economics scholars to make judgments and predictions based on a simple regression analysis. To the authors’ surprise, most of these experts had a hard time accurately deciphering and acting on the results of the kind of analysis they themselves frequently conduct. In particular, the authors found that their description of the findings, which mimicked the industry standard, led to an illusion of predictability -- an erroneous belief that the analyzed outcomes were more predictable than they actually were. The authors argue that simulated experience enables intuitive interpretation of statistical information, thereby communicating analytical results even to decision makers who are not knowledgeable about statistics.