Forecasting and Management of Technology, Second Edition

12.4 Learning from Past Forecasts and Assessments

Judgment can be strengthened by examining the past to improve present performance in anticipating the future. Thus, it is instructive to examine the results of past forecasts to see if they were successful in limiting uncertainty. Also see Section 2.1.4 for common forecasting errors.

Reviewing old forecasts shows that many were too ambiguous to tell if they proved to be accurate. However, George Wise (1976) gathered a large number of forecasts that he could judge to be right or wrong and sorted these into 18 technological areas (e.g., computers, factory automation, new materials). He aggregated predictions for each area, reporting the average percentage right by area. These ranged from 18% right for 22 housing technology forecasts to 78% for 18 new materials forecasts. The median by area was 45% right; the interquartile range was 38% to 51%. That is, one-quarter of the areas fared worse than 38% and one-quarter did better than 51%. This rough estimate helps put forecasting expectations into perspective. They are far short of certainty but much better than chance. But remember, technology forecasts are intended to paint a landscape of possible futures, not a portrait of a certain future.

Michael Scriven (1967) draws a useful distinction between formative and summative evaluation. Formative evaluation considers a study while it is underway and can be very helpful in guiding the forecast. Formative procedures often are quite informal—for ...