book

Time Series with PyTorch

Name: Time Series with PyTorch
ISBN: 9781805128182

by Graeme Davidson, Lei Ma

May 2026

Intermediate to advanced

606 pages

16h 23m

English

Packt Publishing

Read now

Unlock full access

Acknowledgments
Preface
Who this book is forWhat this book coversTo get the most out of this bookGet in touch
Time Series for Everyone
Why time series?The early periodThe classical eraThe promise of deep learningFrom language to timeForecasting and performanceSummaryGet this book’s PDF version and more
The Challenge of Time Series
What is time-series data?Missing dataTypes of missing dataSimple imputationsMICE imputationTheoretical foundations of time series analysisPatterns in timeDecompositionAdditive decompositionMultiplicative decompositionAlternative decompositionsExamples of decompositionSeasonality plottingBeyond decomposition patternsDependence in time seriesAutocorrelation functionPartial autocorrelationStationarityPassengers’ mean and variation split checkTests for stationarityApplying KPSSTime series structuresUnivariateMultivariateMultivariablePanel dataApplication of structures to modelsSummaryRecommended reading
Evaluating Time-Series Models
Why we evaluateUsing validation to manage complexityPartitioning dataIntroducing the datasetACF and PACF plotsConducting residual analysisFixed-origin designTrain-test splitTrain-validation-test splitCross-validationExpanding windowsRolling windowsError metricsIntrinsic metricsAbsolute errorAbsolute percentage error (APE)Symmetric Mean Absolute Percentage Error (sMAPE)Squared errorVariationsExtrinsic metricsGrouped error metricsMultivariate forecastingWhich to use and whenLength of evaluationData leakageTransformation-based leakageLookforward biasPanel data leakageSummaryRecommended readingGet this book’s PDF version and more
PyTorch Fundamentals
Introduction to PyTorchWorking with tensorsStrideBasic tensor constructionUnderstanding computational graphsChain rule revisitedBackpropagation calculationsA brief look at autogradPracticing PyTorch fundamentalsPure PyTorch neural networkLightning NNSummaryRecommended readingJoin our community on discord
Simple Neural Architecture
Basic structures of neural networksArtificial neuronsNeural netsFrom scalars to matricesForward propagationLoss calculationBackpropagationBuilding a neural networkSummaryRecommended readingGet this book’s PDF version and more
Optimization
Optimization costsOptimization of neural networksUnderstanding activation functionsTaxonomyLayer-wise activation functionsFixed-shape activation functionsSigmoidTanhReLUTrainable activation functionsComparing activation functionsHidden layersGradient descent, loss functions, and regularization techniquesLoss functionsOptimizing hyperparametersLearning rate schedulingEpochsBatch sizeDropoutWeight decayData splitting for neural networksApproaches to splittingHyperparameter tuningSummaryRecommended readingJoin our community on discord
Conformal Prediction
Uncertainty quantificationUnderstanding conformal predictionIntuition behind conformal predictionImproving conformal intervalsConformal intervals with quartilesMathematical notationApproaches to splittingConformalized regressionConformalized forecastingWorking with EnbPIApplying EnbPI to our PyTorch modelEvaluating conformal interval qualitySummaryRecommended readingGet this book’s PDF version and more
Recurrent Neural Networks
Time-series data and statesSwing angle forecastingWhy recurrent models helpIntroducing RNNsSimple RNNTheoretical backgroundRNNs and differential equationsImplementationIntroducing LSTMIntroducing GRUsStackingForecasting using RNNsSummaryRecommended reading

Transformers
The vanilla transformerEncoder-decoderAttention mechanismKnowledge of temporal informationImplementation of transformersThe M5 competitionSummaryRecommended readingGet this book’s PDF version and more
Other Neural Structures
Setting up our M5 datasetExploring some neural network familiesMultilayer perceptronN-BEATSN-HITSConvolutional Neural NetworksGraph neural networksKolmogorov–Arnold networksFoundation modelsSummaryRecommended reading
Transfer Learning and Global Modelling
What is transfer learning?Types of transfer learningApplications of transfer learningStructuring time series data for transfer learningTemporal windowing and sequential organizationScale normalization and feature transformationTemporal feature engineeringGlobal modelingBuilding GFMs with PyTorchFeature integration mechanismsData organization and batchingTraining and balanced samplingOptimizers, learning rate scheduling, and batchingTransfer learning with NeuralForecastData preparationStatistical baselinesDecision-tree global modelsNeural network GFMsPretraining with external dataSummaryRecommended readingGet this book’s PDF version and more
Synthetic Time Series Data
The data-generating processData-driven synthetic time seriesGenerating time series data using variational autoencodersImplementing TimeVAEImplementing the modelTraining the modelGenerating synthetic dataBeyond VAESummaryRecommended readingJoin our community on discord
Diffusion Models
Introduction to probabilistic forecastingDiffusion as a real-world phenomenonSimulating the diffusion processNaive diffusion model applied on time series dataDenoising diffusion modelSummaryRecommended readingGet this book’s PDF version and more
Time Series Classification
Types of time-series comparisonsDistance-based measuresNormsDynamic time warpingRetrieving alignment pathsApplied distancesElastic ensembleProximity forestComparing distance methodsFeature-based algorithmsFeatures built-inInterpreting featuresBuilding featuresShapeletsDictionary-based classificationConvolution-based classificationDeep learning approachesConvolutional neural networksResNetInceptionTimeH-InceptionTimeLiteTimeSummaryRecommended readingJoin our community on discord
Time Series Clustering
What is clustering?Preprocessing for time series clusteringDefining time series clusteringThe TSCL pipelineClustering approachesDistance-basedDistribution-basedSubsequence-basedRepresentation learningDeep-learning approaches to TSCLTwo-stage pipelineEnd-to-end neural clusteringApplying featurizationData preparationFeature extractionScalingSelecting the number of clustersBuilding GFMs with cluster labelsDenoising with an autoencoderAutoencoder hyperparametersEvaluating clustering resultsSummaryRecommended readingGet this book’s PDF version and more
Embeddings for Time Series
A dynamical systems perspectiveLearned embeddingsThe diversity of time series embeddingsSummaryRecommended readingJoin our community on discord
Supervised and Unsupervised Anomaly Detection
What is an outlier or anomaly?Evaluating TSAD systemsRule-based approachesAdaptive thresholdsStatistical profilingBuilding profilesResiduals as the detection signalThe One-Class gaussianIsolation forestIsolation forest – unsupervisedStandard isolation forest on residualsExtended isolation forestLocal outlier factorMatrix profileApplication to residualsMultidimensional matrix profileChoosing the window lengthMatrix profile vs. Point-Based methodsAlternative approachesAB-Join: comparing against a reference periodWindow-Overlap evaluationWhere matrix profile fitsA comparative summarySupervised anomaly detectionThe label quality problemClass imbalanceGradient boosting on residual featuresThreshold adjustmentForecast-First anomaly detectionThe pipelineThe iteration loopSupervised extensionComposability in practiceLibrary ecosystemSummaryRecommended readingGet this book’s PDF version and more
Self-Supervised Learning for Time Series
Why do we need representations of time series?Self-supervised learningContrastive predictive codingData preparationModel implementationTraining preparationTraining and evaluationSummaryRecommended readingJoin our community on discord
Unlock Your Exclusive Benefits
Unlock this Book’s Free Benefits in 3 Easy StepsStep 1Step 2Step 3Need Help
Other Books You May Enjoy
Index

Content preview from Time Series with PyTorch

9 Transformers

“… the Transformer, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output.” From Attention Is All You Need

In this new Generative AI era, most of us have heard about the famous transformer model, or the T in ChatGPT. In the seminal paper Attention is All You Need, the legendary transformer architecture was born (Vaswani et al. 2017). The transformer model was initially invented for language tasks. Since time series data and language are quite similar due to their dependency on sequential positions, models that work on natural language processing are mostly tested on time series forecasting tasks too. In fact, companies such as Zalando ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781805128182

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Time Series with PyTorch

by Graeme Davidson, Lei Ma

9

Transformers

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.