book

Modern Time Series Forecasting with Python - Second Edition

Name: Modern Time Series Forecasting with Python - Second Edition
ISBN: 9781835883181

by Manu Joseph, Jeffrey Tackes

October 2024

Intermediate to advanced

660 pages

18h 51m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book coversTo get the most out of this bookSetting up an environmentUsing Anaconda/Miniconda/MambaUsing Python Virtual environments and pipWhat to do when environment creation throws an error?Download the dataGet in touchMaking the Most Out of This Book – Get to Know Your Free Benefits
Part 1: Getting Familiar with Time Series
Introducing Time Series
Technical requirementsWhat is a time series?Types of time seriesMain areas of application for time series analysisData-generating process (DGP)Generating synthetic time seriesWhite and red noiseCyclical or seasonal signalsAutoregressive signalsMix and matchStationary and non-stationary time seriesChange in mean over timeChange in variance over timeWhat can we forecast?Forecasting terminologySummaryFurther reading
Acquiring and Processing Time Series Data
Technical requirementsUnderstanding the time series datasetPreparing a data modelpandas datetime operations, indexing, and slicing—a refresherConverting the date columns into pd.Timestamp/DatetimeIndexUsing the .dt accessor and datetime propertiesIndexing and slicingCreating date sequences and managing date offsetsHandling missing dataConverting the half-hourly block-level data (hhblock) into time series dataCompact, expanded, and wide forms of dataEnforcing regular intervals in time seriesConverting the London Smart Meters dataset into a time series formatExpanded formCompact formMapping additional informationSaving and loading files to diskHandling longer periods of missing dataImputing with the previous dayHourly average profileThe hourly average for each weekdaySeasonal interpolationSummary
Analyzing and Visualizing Time Series Data
Technical requirementsComponents of a time seriesThe trend componentThe seasonal componentThe cyclical componentThe irregular componentVisualizing time series dataLine chartsSeasonal plotsSeasonal box plotsCalendar heatmapsAutocorrelation plotDecomposing a time seriesDetrendingMoving averagesLOESSDeseasonalizingPeriod adjusted averagesFourier seriesImplementationsseasonal_decompose from statsmodelSeasonality and trend decomposition using LOESS (STL)Fourier decompositionMultiple seasonality decomposition using LOESS (MSTL)Detecting and treating outliersStandard deviationIQRIsolation ForestExtreme studentized deviate (ESD) and seasonal ESD (S-ESD)Treating outliersSummaryReferencesFurther reading
Setting a Strong Baseline Forecast
Technical requirementsSetting up a test harnessCreating holdout (test) and validation datasetsChoosing an evaluation metricGenerating strong baseline forecastsNaïve forecastMoving average forecastSeasonal naive forecastExponential smoothingAutoRegressive Integrated Moving Average (ARIMA)Theta forecastTBATSBox-Cox transformationExponentially smoothed trendSeasonal decomposition using Fourier series (trigonometric seasonality)ARMAParameter optimizationMultiple Seasonal-Trend decomposition using LOESS (MSTL)Evaluating the baseline forecastsAssessing the forecastability of a time seriesCoefficient of variationResidual variabilityEntropy-based measuresSpectral entropyKaboudan metricSummaryReferencesFurther reading
Part 2: Machine Learning for Time Series
Time Series Forecasting as Regression
Understanding the basics of machine learningSupervised machine learning tasksOverfitting and underfittingHyperparameters and validation setsTime series forecasting as regressionTime delay embeddingTemporal embeddingGlobal forecasting models—a paradigm shiftSummaryReferencesFurther reading
Feature Engineering for Time Series Forecasting
Technical requirementsUnderstanding feature engineeringAvoiding data leakageSetting a forecast horizonTime delay embeddingLags or backshiftRolling window aggregationsSeasonal rolling window aggregationsExponentially weighted moving average (EWMA)Temporal embeddingCalendar featuresTime elapsedFourier termsSummary
Target Transformations for Time Series Forecasting
Technical requirementsDetecting non-stationarity in time seriesDetecting and correcting for unit rootsUnit rootsThe Augmented Dickey-Fuller (ADF) testDifferencing transformDetecting and correcting for trendsDeterministic and stochastic trendsKendall’s TauMann-Kendall test (M-K test)Detrending transformDetecting and correcting for seasonalityDetecting seasonalityDeseasonalizing transformDetecting and correcting for heteroscedasticityDetecting heteroscedasticityLog transformBox-Cox transformationAutoML approach to target transformationSummaryReferencesFurther reading

Forecasting Time Series with Machine Learning Models
Technical requirementsTraining and predicting with machine learning modelsGenerating single-step forecast baselinesStandardized code to train and evaluate machine learning modelsFeatureConfigMissingValueConfigModelConfigMLForecastThe fit functionThe predict functionThe feature_importance functionHelper functions for evaluating modelsLinear regressionRegularized linear regressionRegularization–a geometric perspectiveDecision treesRandom forestGradient boosting decision treesTraining and predicting for multiple householdsUsing AutoStationaryTransformerSummaryReferencesFurther reading
Ensembling and Stacking
Technical requirementsCombining forecastsBest fitMeasures of central tendencySimple hill climbingStochastic hill climbingSimulated annealingOptimal weighted ensembleStacking and blendingSummaryReferencesFurther reading
Global Forecasting Models
Technical requirementsWhy Global Forecasting Models?Sample sizeCross-learningMulti-task learningEngineering complexityCreating GFMsStrategies to improve GFMsIncreasing memoryAdding more lag featuresAdding rolling featuresAdding EWMA featuresUsing time series meta-featuresOrdinal encoding and one-hot encodingFrequency encodingTarget mean encodingLightGBM’s native handling of categorical featuresTuning hyperparametersGrid searchRandom searchBayesian optimizationPartitioningRandom partitionJudgmental partitioningAlgorithmic partitioningInterpretabilitySummaryReferencesFurther reading
Part 3: Deep Learning for Time Series
Introduction to Deep Learning
Technical requirementsWhat is deep learning and why now?Why now?Increase in compute availabilityIncrease in data availabilityWhat is deep learning?Perceptron, the first neural networkComponents of a deep learning systemRepresentation learningLinear transformationActivation functionsSigmoidHyperbolic tangent (tanh)Rectified linear units and variantsOutput activation functionsSoftmaxLoss functionForward and backward propagationGradient descentSummaryReferencesFurther reading
Building Blocks of Deep Learning for Time Series
Technical requirementsUnderstanding the encoder-decoder paradigmFeed-forward networksRecurrent neural networksRNN architectureRNN in PyTorchLong short-term memory (LSTM) networksLSTM architectureLSTM in PyTorchGated recurrent unit (GRU)GRU architectureGRU in PyTorchConvolution networksConvolutionPadding, stride, and dilationsConvolution in PyTorchSummaryReferencesFurther reading
Common Modeling Patterns for Time Series
Technical requirementsTabular regressionSingle-step-ahead recurrent neural networksSequence-to-sequence (Seq2Seq) modelsRNN-to-fully connected networkRNN-to-RNNSummaryReferenceFurther reading
Attention and Transformers for Time Series
Technical requirementsWhat is attention?The generalized attention modelAlignment functionsDot productScaled dot product attentionGeneral attentionAdditive/concat attentionThe distribution functionForecasting with sequence-to-sequence models and attentionTransformers—Attention is all you needAttention is all you needSelf-attentionMulti-headed attentionPositional encodingPosition-wise feed-forward layerEncoderDecoderTransformers in time seriesForecasting with TransformersSummaryReferencesFurther reading
Strategies for Global Deep Learning Forecasting Models
Technical requirementsCreating global deep learning forecasting modelsPreprocessing the dataUnderstanding TimeSeriesDataset from PyTorch ForecastingInitializing TimeSeriesDatasetCreating the dataloaderVisualizing how the dataloader worksBuilding the first global deep learning forecasting modelDefining our first RNN modelInitializing the RNN modelTraining the RNN modelForecasting with the trained modelUsing time-varying informationUsing static/meta informationOne-hot encoding and why it is not idealEmbedding vectors and dense representationsDefining a model with categorical featuresUsing the scale of the time seriesBalancing the sampling procedureVisualizing the data distributionTweaking the sampling procedureUsing and visualizing the dataloader with WeightedRandomSamplerSummaryFurther reading
Specialized Deep Learning Architectures for Forecasting
Technical requirementsThe need for specialized architecturesIntroduction to NeuralForecastCommon parameters and configurations“Auto” modelsExogenous featuresNeural Basis Expansion Analysis for Interpretable Time Series Forecasting (N-BEATS)The architecture of N-BEATSBlocksStacksThe overall architectureBasis functions and interpretabilityForecasting with N-BEATSInterpreting N-BEATS forecastingNeural Basis Expansion Analysis for Interpretable Time Series Forecasting with Exogenous Variables (N-BEATSx)Handling exogenous variablesExogenous blocksNeural Hierarchical Interpolation for Time Series Forecasting (N-HiTS)The Architecture of N-HiTSMulti-rate data samplingHierarchical interpolationSynchronizing the input sampling and output interpolationForecasting with N-HiTSAutoformerThe architecture of the Autoformer modelUniform Input RepresentationGenerative-style decoderDecomposition architectureAuto-correlation mechanismForecasting with AutoformerLTSF-Linear family of modelsLinearD-LinearN-LinearForecasting with the LTSF-Linear familyPatch Time Series Transformer (PatchTST)The architecture of the PatchTST modelPatchingChannel independenceForecasting with PatchTSTiTransformerThe architecture of iTransformerForecasting with iTransformerTemporal Fusion Transformer (TFT)The architecture of TFTLocality Enhancement Seq2Seq layerTemporal fusion decoderGated residual networksVariable selection networksForecasting with TFTInterpreting TFTTSMixerThe architecture of the TSMixer modelMixer LayerTemporal Projection LayerTSMixerx—TSMixer with auxiliary informationForecasting with TSMixer and TSMixerxTime Series Dense Encoder (TiDE)The architecture of the TiDE modelResidual blockEncoderDecoderForecasting with TiDESummaryReferencesFurther reading
Probabilistic Forecasting and More
Probabilistic forecastingTypes of Predictive UncertaintyWhat are probabilistic forecasts and Prediction Intervals?Confidence levels, error rates, and quantilesMeasuring the goodness of prediction intervalsProbability Density Function (PDF)Forecasting with PDF—machine learning modelsForecasting with PDF—deep learning modelsQuantile functionForecasting with quantile loss (machine learning)Forecasting with quantile loss (deep learning)Monte Carlo DropoutCreating a custom model in neuralforecastForecasting with MC Dropout (neuralforecast)Conformal PredictionConformal Prediction for classificationConformal Prediction for regressionConformalized Quantile RegressionConformalizing uncertainty estimatesExchangeability in Conformal Prediction and time series forecastingForecasting with Conformal PredictionRoad less traveled in time series forecastingIntermittent or sparse demand forecastingInterpretabilityCold-start forecastingHierarchical forecastingSummaryReferencesFurther reading
Part 4: Mechanics of Forecasting
Multi-Step Forecasting
Why multi-step forecasting?Standard notationRecursive strategyTraining regimeForecasting regimeDirect strategyTraining regimeForecasting regimeThe Joint strategyTraining regimeForecasting regimeHybrid strategiesDirRec strategyTraining regimeForecasting regimeIterative block-wise direct strategyTraining regimeForecasting regimeRectify strategyTraining regimeForecasting regimeRecJointTraining regimeForecasting regimeHow to choose a multi-step forecasting strategySummaryReferences
Evaluating Forecast Errors—A Survey of Forecast Metrics
Technical requirementsTaxonomy of forecast error measuresIntrinsic metricsAbsolute errorSquared errorPercent errorSymmetric errorOther intrinsic metricsExtrinsic metricsRelative errorScaled errorOther extrinsic metricsInvestigating the error measuresLoss curves and complementarityAbsolute errorSquared errorPercent errorSymmetric errorExtrinsic errorsBias toward over- or under-forecastingExperimental study of the error measuresUsing Spearman’s rank correlationGuidelines for choosing a metricSummaryReferencesFurther reading
Evaluating Forecasts—Validation Strategies
Technical requirementsModel validationHoldout strategiesWindow strategyCalibration strategySampling strategyCross-validation strategiesChoosing a validation strategyValidation strategies for datasets with multiple time seriesSummaryReferencesFurther reading
Other Books You May Enjoy
Index

Content preview from Modern Time Series Forecasting with Python - Second Edition

14 Attention and Transformers for Time Series

In the previous chapter, we rolled up our sleeves and implemented a few deep learning (DL) systems for time series forecasting. We used the common building blocks we discussed in Chapter 12, Building Blocks of Deep Learning for Time Series, put them together in an encoder-decoder architecture, and trained them to produce the forecast we desired.

Now, let’s talk about another key concept in DL that has taken the field by storm over the past few years—attention. Attention has a long-standing history, which has culminated in it being one of the most sought-after tools in the DL toolkit. This chapter takes you on a journey to understand attention and transformer models from the ground up from a theoretical ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Modern Time Series Forecasting with Python

Publisher Resources

ISBN: 9781835883181

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Modern Time Series Forecasting with Python - Second Edition

by Manu Joseph, Jeffrey Tackes

14

Attention and Transformers for Time Series

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.