book

Modern Time Series Forecasting with Python - Second Edition

by Manu Joseph, Jeffrey Tackes

October 2024

Intermediate to advanced

660 pages

18h 51m

English

Packt Publishing

Read now

Unlock full access

Who this book is forWhat this book coversTo get the most out of this bookSetting up an environmentUsing Anaconda/Miniconda/MambaUsing Python Virtual environments and pipWhat to do when environment creation throws an error?Download the dataGet in touchMaking the Most Out of This Book – Get to Know Your Free Benefits
Technical requirementsWhat is a time series?Types of time seriesMain areas of application for time series analysisData-generating process (DGP)Generating synthetic time seriesWhite and red noiseCyclical or seasonal signalsAutoregressive signalsMix and matchStationary and non-stationary time seriesChange in mean over timeChange in variance over timeWhat can we forecast?Forecasting terminologySummaryFurther reading
Technical requirementsUnderstanding the time series datasetPreparing a data modelpandas datetime operations, indexing, and slicing—a refresherConverting the date columns into pd.Timestamp/DatetimeIndexUsing the .dt accessor and datetime propertiesIndexing and slicingCreating date sequences and managing date offsetsHandling missing dataConverting the half-hourly block-level data (hhblock) into time series dataCompact, expanded, and wide forms of dataEnforcing regular intervals in time seriesConverting the London Smart Meters dataset into a time series formatExpanded formCompact formMapping additional informationSaving and loading files to diskHandling longer periods of missing dataImputing with the previous dayHourly average profileThe hourly average for each weekdaySeasonal interpolationSummary
Technical requirementsComponents of a time seriesThe trend componentThe seasonal componentThe cyclical componentThe irregular componentVisualizing time series dataLine chartsSeasonal plotsSeasonal box plotsCalendar heatmapsAutocorrelation plotDecomposing a time seriesDetrendingMoving averagesLOESSDeseasonalizingPeriod adjusted averagesFourier seriesImplementationsseasonal_decompose from statsmodelSeasonality and trend decomposition using LOESS (STL)Fourier decompositionMultiple seasonality decomposition using LOESS (MSTL)Detecting and treating outliersStandard deviationIQRIsolation ForestExtreme studentized deviate (ESD) and seasonal ESD (S-ESD)Treating outliersSummaryReferencesFurther reading
Technical requirementsSetting up a test harnessCreating holdout (test) and validation datasetsChoosing an evaluation metricGenerating strong baseline forecastsNaïve forecastMoving average forecastSeasonal naive forecastExponential smoothingAutoRegressive Integrated Moving Average (ARIMA)Theta forecastTBATSBox-Cox transformationExponentially smoothed trendSeasonal decomposition using Fourier series (trigonometric seasonality)ARMAParameter optimizationMultiple Seasonal-Trend decomposition using LOESS (MSTL)Evaluating the baseline forecastsAssessing the forecastability of a time seriesCoefficient of variationResidual variabilityEntropy-based measuresSpectral entropyKaboudan metricSummaryReferencesFurther reading
Understanding the basics of machine learningSupervised machine learning tasksOverfitting and underfittingHyperparameters and validation setsTime series forecasting as regressionTime delay embeddingTemporal embeddingGlobal forecasting models—a paradigm shiftSummaryReferencesFurther reading
Technical requirementsUnderstanding feature engineeringAvoiding data leakageSetting a forecast horizonTime delay embeddingLags or backshiftRolling window aggregationsSeasonal rolling window aggregationsExponentially weighted moving average (EWMA)Temporal embeddingCalendar featuresTime elapsedFourier termsSummary
Technical requirementsDetecting non-stationarity in time seriesDetecting and correcting for unit rootsUnit rootsThe Augmented Dickey-Fuller (ADF) testDifferencing transformDetecting and correcting for trendsDeterministic and stochastic trendsKendall’s TauMann-Kendall test (M-K test)Detrending transformDetecting and correcting for seasonalityDetecting seasonalityDeseasonalizing transformDetecting and correcting for heteroscedasticityDetecting heteroscedasticityLog transformBox-Cox transformationAutoML approach to target transformationSummaryReferencesFurther reading

Technical requirementsTraining and predicting with machine learning modelsGenerating single-step forecast baselinesStandardized code to train and evaluate machine learning modelsFeatureConfigMissingValueConfigModelConfigMLForecastThe fit functionThe predict functionThe feature_importance functionHelper functions for evaluating modelsLinear regressionRegularized linear regressionRegularization–a geometric perspectiveDecision treesRandom forestGradient boosting decision treesTraining and predicting for multiple householdsUsing AutoStationaryTransformerSummaryReferencesFurther reading
Technical requirementsCombining forecastsBest fitMeasures of central tendencySimple hill climbingStochastic hill climbingSimulated annealingOptimal weighted ensembleStacking and blendingSummaryReferencesFurther reading
Technical requirementsWhy Global Forecasting Models?Sample sizeCross-learningMulti-task learningEngineering complexityCreating GFMsStrategies to improve GFMsIncreasing memoryAdding more lag featuresAdding rolling featuresAdding EWMA featuresUsing time series meta-featuresOrdinal encoding and one-hot encodingFrequency encodingTarget mean encodingLightGBM’s native handling of categorical featuresTuning hyperparametersGrid searchRandom searchBayesian optimizationPartitioningRandom partitionJudgmental partitioningAlgorithmic partitioningInterpretabilitySummaryReferencesFurther reading
Technical requirementsWhat is deep learning and why now?Why now?Increase in compute availabilityIncrease in data availabilityWhat is deep learning?Perceptron, the first neural networkComponents of a deep learning systemRepresentation learningLinear transformationActivation functionsSigmoidHyperbolic tangent (tanh)Rectified linear units and variantsOutput activation functionsSoftmaxLoss functionForward and backward propagationGradient descentSummaryReferencesFurther reading
Technical requirementsUnderstanding the encoder-decoder paradigmFeed-forward networksRecurrent neural networksRNN architectureRNN in PyTorchLong short-term memory (LSTM) networksLSTM architectureLSTM in PyTorchGated recurrent unit (GRU)GRU architectureGRU in PyTorchConvolution networksConvolutionPadding, stride, and dilationsConvolution in PyTorchSummaryReferencesFurther reading
Technical requirementsTabular regressionSingle-step-ahead recurrent neural networksSequence-to-sequence (Seq2Seq) modelsRNN-to-fully connected networkRNN-to-RNNSummaryReferenceFurther reading
Technical requirementsWhat is attention?The generalized attention modelAlignment functionsDot productScaled dot product attentionGeneral attentionAdditive/concat attentionThe distribution functionForecasting with sequence-to-sequence models and attentionTransformers—Attention is all you needAttention is all you needSelf-attentionMulti-headed attentionPositional encodingPosition-wise feed-forward layerEncoderDecoderTransformers in time seriesForecasting with TransformersSummaryReferencesFurther reading
Technical requirementsCreating global deep learning forecasting modelsPreprocessing the dataUnderstanding TimeSeriesDataset from PyTorch ForecastingInitializing TimeSeriesDatasetCreating the dataloaderVisualizing how the dataloader worksBuilding the first global deep learning forecasting modelDefining our first RNN modelInitializing the RNN modelTraining the RNN modelForecasting with the trained modelUsing time-varying informationUsing static/meta informationOne-hot encoding and why it is not idealEmbedding vectors and dense representationsDefining a model with categorical featuresUsing the scale of the time seriesBalancing the sampling procedureVisualizing the data distributionTweaking the sampling procedureUsing and visualizing the dataloader with WeightedRandomSamplerSummaryFurther reading
Technical requirementsThe need for specialized architecturesIntroduction to NeuralForecastCommon parameters and configurations“Auto” modelsExogenous featuresNeural Basis Expansion Analysis for Interpretable Time Series Forecasting (N-BEATS)The architecture of N-BEATSBlocksStacksThe overall architectureBasis functions and interpretabilityForecasting with N-BEATSInterpreting N-BEATS forecastingNeural Basis Expansion Analysis for Interpretable Time Series Forecasting with Exogenous Variables (N-BEATSx)Handling exogenous variablesExogenous blocksNeural Hierarchical Interpolation for Time Series Forecasting (N-HiTS)The Architecture of N-HiTSMulti-rate data samplingHierarchical interpolationSynchronizing the input sampling and output interpolationForecasting with N-HiTSAutoformerThe architecture of the Autoformer modelUniform Input RepresentationGenerative-style decoderDecomposition architectureAuto-correlation mechanismForecasting with AutoformerLTSF-Linear family of modelsLinearD-LinearN-LinearForecasting with the LTSF-Linear familyPatch Time Series Transformer (PatchTST)The architecture of the PatchTST modelPatchingChannel independenceForecasting with PatchTSTiTransformerThe architecture of iTransformerForecasting with iTransformerTemporal Fusion Transformer (TFT)The architecture of TFTLocality Enhancement Seq2Seq layerTemporal fusion decoderGated residual networksVariable selection networksForecasting with TFTInterpreting TFTTSMixerThe architecture of the TSMixer modelMixer LayerTemporal Projection LayerTSMixerx—TSMixer with auxiliary informationForecasting with TSMixer and TSMixerxTime Series Dense Encoder (TiDE)The architecture of the TiDE modelResidual blockEncoderDecoderForecasting with TiDESummaryReferencesFurther reading
Probabilistic forecastingTypes of Predictive UncertaintyWhat are probabilistic forecasts and Prediction Intervals?Confidence levels, error rates, and quantilesMeasuring the goodness of prediction intervalsProbability Density Function (PDF)Forecasting with PDF—machine learning modelsForecasting with PDF—deep learning modelsQuantile functionForecasting with quantile loss (machine learning)Forecasting with quantile loss (deep learning)Monte Carlo DropoutCreating a custom model in neuralforecastForecasting with MC Dropout (neuralforecast)Conformal PredictionConformal Prediction for classificationConformal Prediction for regressionConformalized Quantile RegressionConformalizing uncertainty estimatesExchangeability in Conformal Prediction and time series forecastingForecasting with Conformal PredictionRoad less traveled in time series forecastingIntermittent or sparse demand forecastingInterpretabilityCold-start forecastingHierarchical forecastingSummaryReferencesFurther reading
Why multi-step forecasting?Standard notationRecursive strategyTraining regimeForecasting regimeDirect strategyTraining regimeForecasting regimeThe Joint strategyTraining regimeForecasting regimeHybrid strategiesDirRec strategyTraining regimeForecasting regimeIterative block-wise direct strategyTraining regimeForecasting regimeRectify strategyTraining regimeForecasting regimeRecJointTraining regimeForecasting regimeHow to choose a multi-step forecasting strategySummaryReferences
Technical requirementsTaxonomy of forecast error measuresIntrinsic metricsAbsolute errorSquared errorPercent errorSymmetric errorOther intrinsic metricsExtrinsic metricsRelative errorScaled errorOther extrinsic metricsInvestigating the error measuresLoss curves and complementarityAbsolute errorSquared errorPercent errorSymmetric errorExtrinsic errorsBias toward over- or under-forecastingExperimental study of the error measuresUsing Spearman’s rank correlationGuidelines for choosing a metricSummaryReferencesFurther reading
Technical requirementsModel validationHoldout strategiesWindow strategyCalibration strategySampling strategyCross-validation strategiesChoosing a validation strategyValidation strategies for datasets with multiple time seriesSummaryReferencesFurther reading

Content preview from Modern Time Series Forecasting with Python - Second Edition

9 Ensembling and Stacking

In the previous chapter, we looked at a few machine learning algorithms and used them to generate forecasts on the London Smart Meters dataset. Now that we have multiple forecasts for all the households in the dataset, how do we come up with a single forecast by choosing or combining these different forecasts? At the end of the day, we can only have one forecast that will be used for planning whatever task for which you are forecasting. That is what we will be doing in this chapter—we will learn how to leverage combinatorial and mathematical optimization to come up with a single forecast.

In this chapter, we will cover the following topics: