Business Forecasting

CHAPTER 2Big Data in Forecasting

A major objective of this book is to provide sober, measured guidance on the practice of business forecasting and the current trends being hyped in the media and vendor marketing. One such trend, starting well before 2015, is the notion of big data, and how (like AI/ML after it), big data will alter forecasting in meaningful and positive ways. But just as we asked about AI/ML in Chapter 1, will big data have this positive effect?

Prior to M4 results in 2018, there was a large body of evidence supporting a simple and parsimonious approach to forecast modeling. Going as far back as the original M-competition in 1979, it was found that even simple and time-worn methods like exponential smoothing could perform on par with the latest, complex innovations – with fewer data and computational requirements. In a comparison of simple versus complex methods in 2015 (Journal of Business Research 68: 1678–1685), Green and Armstrong went so far as to assert that complexity harms accuracy.

It was not until the M4, with the two top performers being hybrids/combinations of ML and traditional statistical methods, that a more positive view of model complexity has emerged. But as we saw in the M4 analysis, this modest improvement in forecast accuracy came at prohibitive cost. These new methods, at least for the moment, have limited practical consequence.

In the two articles (and accompanying commentaries) in this chapter, we find contrasting views of the potential ...