book

Time Series Forecasting Using Foundation Models

Name: Time Series Forecasting Using Foundation Models
Author: Marco Peixeiro
ISBN: 9781633435896

by Marco Peixeiro

November 2025

Intermediate to advanced

256 pages

6h 54m

English

Manning Publications

Read now

Unlock full access

Time Series Forecasting Using Foundation Models
copyright
contents
dedication
preface
acknowledgments
about this book
about the author
about the cover illustration
Part 1 The rise of foundation machine learning models

1 Understanding foundation models
1.1 Defining a foundation model1.2 Exploring the transformer architecture1.2.1 Feeding the encoder1.2.2 Inside the encoder1.2.3 Making predictions1.3 Advantages and disadvantages of foundation models1.3.1 Benefits of foundation forecasting models1.3.2 Drawbacks of foundation forecasting models1.4 Next steps
2 Building a foundation model
2.1 Exploring the architecture of N-BEATS2.1.1 Basis expansion2.2 Architecture of N-BEATS2.2.1 A block in N-BEATS2.2.2 A stack in N-BEATS2.2.3 Assembling N-BEATS2.3 Pretraining our model2.4 Pretraining N-BEATS2.5 Transfer learning with our pretrained model2.6 Fine-tuning our pretrained model2.7 Evaluating each approach2.8 Forecasting at another frequency2.9 Understanding the challenges of building a foundation model2.10 Next steps
Part 2 Foundation models developed for forecasting
3 Forecasting with TimeGPT
3.1 Defining generative pretrained transformers3.2 Exploring TimeGPT3.2.1 Training TimeGPT3.2.2 Quantifying uncertainty in TimeGPT3.3 Forecasting with TimeGPT3.3.1 Initial setup3.3.2 Zero-shot forecasting3.3.3 Performance evaluation3.4 Fine-tuning with TimeGPT3.4.1 Fine-tuning TimeGPT3.4.2 Evaluating the fine-tuned model3.4.3 Controlling the depth of fine-tuning3.5 Forecasting with exogenous variables3.5.1 Preparing the exogenous features3.5.2 Forecasting with exogenous variables3.5.3 Explaining the effect of exogenous features with Shapley values3.5.4 Evaluating forecasts with exogenous features3.6 Cross-validating with TimeGPT3.7 Forecasting on a long horizon with TimeGPT3.8 Detecting anomalies with TimeGPT3.8.1 Detecting anomalies3.8.2 Evaluating anomaly detection3.9 Next steps
4 Zero-shot probabilistic forecasting with Lag-Llama
4.1 Exploring Lag-Llama4.1.1 Viewing the architecture of Lag-Llama4.1.2 Pretraining Lag-Llama4.2 Forecasting with Lag-Llama4.2.1 Setting up Lag-Llama4.2.2 Zero-shot forecasting with Lag-Llama4.2.3 Changing the context length in Lag-Llama4.3 Fine-tuning Lag-Llama4.3.1 Handling initial setup4.3.2 Reading and splitting the data in Colab4.3.3 Launching the fine-tuning procedure4.3.4 Forecasting with a fine-tuned model4.3.5 Evaluating the fine-tuned model4.4 Model comparison table4.5 Next steps
5 Learning the language of time with Chronos
5.1 Discovering the T5 family5.2 Exploring Chronos5.3 Using tokenization in Chronos5.4 Training a model with Chronos5.4.1 Tackling data scarcity with augmentation techniques5.4.2 Examining the pretrained Chronos models5.4.3 Selecting the appropriate Chronos model5.5 Forecasting with Chronos5.5.1 Initial setup5.5.2 Predictions5.6 Cross-validating with Chronos5.6.1 Running cross-validation5.6.2 Evaluating Chronos5.7 Fine-tuning Chronos5.7.1 Performing initial setup5.7.2 Configuring the fine-tuning parameters5.7.3 Launching the fine-tuning procedure5.7.4 Forecasting with a fine-tuned model5.7.5 Evaluating the fine-tuned model5.8 Detecting anomalies with Chronos5.9 Next steps
6 Moirai: A universal forecasting transformer
6.1 Exploring Moirai6.1.1 Viewing the architecture of Moirai6.1.2 Pretraining Moirai6.1.3 Selecting the appropriate model6.2 Discovering Moirai-MoE6.2.1 Patching and embedding6.2.2 Studying the decoder-only transformer6.2.3 Pretraining Moirai-MoE6.3 Forecasting with Moirai6.3.1 Zero-shot forecasting with Moirai6.3.2 Cross-validation with Moirai6.3.3 Forecasting with exogenous features6.4 Detecting anomalies with Moirai6.5 Next steps
7 Deterministic forecasting with TimesFM
7.1 Examining TimesFM7.1.1 Architecture of TimesFM7.1.2 Pretraining TimesFM7.2 Forecasting with TimesFM7.2.1 Zero-shot forecasting with TimesFM7.2.2 Cross-validation with TimesFM7.2.3 Forecasting with exogenous features7.3 Fine-tuning TimesFM and anomaly detection7.4 Next steps
Part 3 Using LLMs for time-series forecasting
8 Forecasting as a language task
8.1 Overview of LLMs and prompting techniques8.1.1 Exploring Flan-T5 and Llama-3.28.1.2 Understanding the basics of prompting8.2 Forecasting with Flan-T58.2.1 Function to forecast with Flan-T58.2.2 Forecast with Flan-T58.3 Cross-validation with Flan-T58.3.1 Running cross-validation8.3.2 Evaluating Flan-T58.4 Forecasting with exogenous features with Flan-T58.4.1 Including exogenous features with Flan-T58.4.2 Extracting future values of exogenous variables8.4.3 Cross-validating with external features8.4.4 Evaluating Flan-T5 forecasts with exogenous features8.5 Detecting anomalies with Flan-T58.5.1 Defining a function for anomaly detection with Flan-T58.5.2 Running anomaly detection8.6 Forecasting with Llama-3.28.6.1 Performing initial setup8.6.2 Creating a function to forecast via API call8.6.3 Making predictions8.7 Cross-validating with Llama-3.28.8 Detecting anomalies with Llama-3.28.8.1 Modifying the system prompt8.8.2 Defining a function for anomaly detection8.8.3 Running anomaly detection with Llama-3.28.8.4 Evaluating anomaly detection8.9 Next steps
9 Reprogramming an LLM for forecasting
9.1 Discovering Time-LLM9.1.1 Patch reprogramming9.1.2 Discovering Prompt-as-Prefix9.1.3 Making predictions9.2 Forecasting with Time-LLM9.2.1 Performing initial setup9.2.2 Generating forecasts9.3 Cross-validating with Time-LLM9.4 Evaluating Time-LLM9.5 Detecting anomalies with Time-LLM9.5.1 Detecting anomalies9.5.2 Evaluating anomaly detection9.6 Next steps
Part 4 Capstone project
10 Capstone project: Forecasting daily visits to a blog
10.1 Introducing the use case10.2 Walking through the project10.2.1 Setting the constants10.2.2 Forecasting with a seasonal naïve model10.2.3 Forecasting with ARIMA10.2.4 Forecasting with TimeGPT10.2.5 Forecasting with Chronos10.2.6 Forecasting with Moirai10.2.7 Forecasting with TimesFM10.2.8 Forecasting with Time-LLM10.2.9 Evaluating all models10.3 Staying up to date
references

Content preview from Time Series Forecasting Using Foundation Models

2 Building a foundation model

This chapter covers

Exploring the architecture of N-BEATS
Pretraining our own model for transfer learning
Fine-tuning a pretrained model
Understanding the challenges of building a foundation model

In chapter 1, we defined foundation models, discovered their advantages and drawbacks, and explored the transformer architecture, which powers the vast majority of foundation models that we will encounter throughout this book. Before we delve into those advanced models, let’s build our own tiny foundation model to understand the concepts surrounding foundation models and appreciate the challenges researchers overcome to build them.

Although technically, many deep learning models can be pretrained and used for transfer ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633435896Publisher Support Other Publisher Website Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Time Series Forecasting Using Foundation Models

by Marco Peixeiro