book

Modern Time Series Forecasting with Python

by Manu Joseph

November 2022

Beginner to intermediate

552 pages

16h 4m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewers
Who this book is forWhat this book coversTo get the most out of this bookSetting up an environmentDownload the dataDownload the example code filesDownload the color imagesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Technical requirementsWhat is a time series?Types of time seriesMain areas of application for time series analysisData-generating process (DGP)Generating synthetic time seriesStationary and non-stationary time seriesWhat can we forecast?Forecasting terminologySummaryFurther reading
Technical requirementsUnderstanding the time series datasetPreparing a data modelpandas datetime operations, indexing, and slicing  – a refresherConverting the date columns into pd.Timestamp/DatetimeIndexUsing the .dt accessor and datetime propertiesSlicing and indexingCreating date sequences and managing date offsetsHandling missing dataConverting the half-hourly block-level data (hhblock) into time series dataCompact, expanded, and wide forms of dataEnforcing regular intervals in time seriesConverting the London Smart Meters dataset into a time series formatMapping additional informationSaving and loading files to diskHandling longer periods of missing dataImputing with the previous dayHourly average profileThe hourly average for each weekdaySeasonal interpolationSummary
Technical requirementsComponents of a time seriesThe trend componentThe seasonal componentThe cyclical componentThe irregular componentVisualizing time series dataLine chartsSeasonal plotsSeasonal box plotsCalendar heatmapsAutocorrelation plotDecomposing a time seriesDetrendingDeseasonalizingImplementationsDetecting and treating outliersStandard deviationInterquartile range (IQR)Isolation ForestExtreme studentized deviate (ESD) and seasonal ESD (S-ESD)Treating outliersSummaryReferencesFurther reading
Technical requirementsSetting up a test harnessCreating holdout (test) and validation datasetsChoosing an evaluation metricGenerating strong baseline forecastsNaïve forecastMoving average forecastSeasonal naive forecastExponential smoothing (ETS)ARIMATheta ForecastFast Fourier Transform forecastEvaluating the baseline forecastsAssessing the forecastability of a time seriesCoefficient of Variation (CoV)Residual variability (RV)Entropy-based measuresKaboudan metricSummaryReferencesFurther reading
Understanding the basics of machine learningSupervised machine learning tasksOverfitting and underfittingHyperparameters and validation sets Time series forecasting as regressionTime delay embeddingTemporal embeddingGlobal forecasting models – a paradigm shiftSummaryReferencesFurther reading
Technical requirementsFeature engineeringAvoiding data leakageSetting a forecast horizonTime delay embeddingLags or backshiftRolling window aggregationsSeasonal rolling window aggregationsExponentially weighted moving averages (EWMA)Temporal embeddingCalendar featuresTime elapsedFourier termsSummary

Technical requirementsHandling non-stationarity in time seriesDetecting and correcting for unit rootsUnit rootsThe Augmented Dickey-Fuller (ADF) testDifferencing transformDetecting and correcting for trendsDeterministic and stochastic trendsKendall’s TauMann-Kendall test (M-K test)Detrending transformDetecting and correcting for seasonalityDetecting seasonalityDeseasonalizing transformDetecting and correcting for heteroscedasticityDetecting heteroscedasticityLog transformBox-Cox transformAutoML approach to target transformationSummaryReferencesFurther reading
Technical requirementsTraining and predicting with machine learning modelsGenerating single-step forecast baselinesStandardized code to train and evaluate machine learning modelsFeatureConfigMissingValueConfigModelConfigMLForecastHelper functions for evaluating modelsLinear regressionRegularized linear regressionDecision treesRandom forestGradient boosting decision treesTraining and predicting for multiple householdsUsing AutoStationaryTransformerSummaryReferencesFurther reading
Technical requirementsCombining forecastsBest fitMeasures of central tendencySimple hill climbingStochastic hill climbingSimulated annealingOptimal weighted ensembleStacking or blendingSummaryReferencesFurther reading
Technical requirementsWhy Global Forecasting Models (GFMs)?Sample sizeCross-learningMulti-task learningEngineering complexityCreating GFMsStrategies to improve GFMsIncreasing memoryUsing time series meta-featuresTuning hyperparametersPartitioningBonus – interpretabilitySummaryReferencesFurther reading
Technical requirementsWhat is deep learning and why now?Why now?What is deep learning?Perceptron – the first neural networkComponents of a deep learning systemRepresentation learningLinear transformationActivation functionsOutput activation functionsLoss functionForward and backward propagationSummaryReferencesFurther reading
Technical requirementsUnderstanding the encoder-decoder paradigmFeed-forward networksRecurrent neural networksThe RNN layer in PyTorchLong short-term memory (LSTM) networksThe LSTM layer in PyTorchGated recurrent unit (GRU)The GRU layer in PyTorchConvolution networksConvolutionPadding, stride, and dilationsThe convolution layer in PyTorchSummaryReferencesFurther reading
Technical requirementsTabular regressionSingle-step-ahead recurrent neural networksSequence-to-sequence (Seq2Seq) modelsRNN-to-fully connected networkRNN-to-RNNSummaryReferenceFurther reading
Technical requirementsWhat is attention?The generalized attention modelAlignment functionsThe distribution functionForecasting with sequence-to-sequence models and attentionTransformers – Attention is all you needAttention is all you needTransformers in time seriesForecasting with TransformersSummaryReferencesFurther reading
Technical requirementsCreating global deep learning forecasting modelsPreprocessing the dataUnderstanding TimeSeriesDataset from PyTorch ForecastingBuilding the first global deep learning forecasting modelUsing time-varying informationUsing static/meta informationOne-hot encoding and why it is not idealEmbedding vectors and dense representationsDefining a model with categorical featuresUsing the scale of the time seriesBalancing the sampling procedureVisualizing the data distributionTweaking the sampling procedureUsing and visualizing the dataloader with WeightedRandomSamplerSummaryFurther reading
Technical requirementsThe need for specialized architecturesNeural Basis Expansion Analysis for Interpretable Time Series Forecasting (N-BEATS)The architecture of N-BEATSForecasting with N-BEATSInterpreting N-BEATS forecastingNeural Basis Expansion Analysis for Interpretable Time Series Forecasting with Exogenous Variables (N-BEATSx)Handling exogenous variablesExogenous blocksNeural Hierarchical Interpolation for Time Series Forecasting (N-HiTS)The Architecture of N-HiTSForecasting with N-HiTSInformerThe architecture of the Informer modelForecasting with the Informer modelAutoformerThe architecture of the Autoformer modelForecasting with AutoformerTemporal Fusion Transformer (TFT)The Architecture of TFTForecasting with TFTInterpreting TFTInterpretabilityProbabilistic forecastingProbability Density Function (PDF)Quantile functionsOther approachesSummaryReferencesFurther reading
Why multi-step forecasting?Recursive strategyTraining regimeForecasting regimeDirect strategyTraining regimeForecasting regimeJoint strategyTraining regimeForecasting regimeHybrid strategiesDirRec StrategyIterative block-wise direct strategyRectify strategyRecJointHow to choose a multi-step forecasting strategy?SummaryReferences
Technical requirementsTaxonomy of forecast error measuresIntrinsic metricsExtrinsic metricsInvestigating the error measuresLoss curves and complementarityBias towards over- or under-forecastingExperimental study of the error measuresUsing Spearman’s rank correlationGuidelines for choosing a metricSummaryReferencesFurther reading
Technical requirementsModel validationHoldout strategiesWindow strategyCalibration strategySampling strategyCross-validation strategiesChoosing a validation strategyValidation strategies for datasets with multiple time seriesSummaryReferencesFurther reading
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Modern Time Series Forecasting with Python

14 Attention and Transformers for Time Series

In the previous chapter, we rolled up our sleeves and implemented a few deep learning (DL) systems for time series forecasting. We used the common building blocks we discussed in Chapter 12, Building Blocks of Deep Learning for Time Series, put them together in an encoder-decoder architecture, and trained them to produce the forecast we desired.

Now, let’s talk about another key concept in DL that has taken the field by storm over the past few years—attention. Attention has a long-standing history, which has culminated in it being one of the most sought-after tools in the DL toolkit. This chapter takes you on a journey to understand attention and transformer models from the ground up from a theoretical ...