Skip to Content
View all events

Continual Learning Beyond the Context Window

Published by O'Reilly Media, Inc.

Intermediate to advanced content levelIntermediate to advanced

Fine-tune embedding models and SLMs from agent memory traces

What you’ll learn and how you can apply it

  • Curate fine-tuning datasets from high-signal agent memory traces, including contrastive pairs for embedding training and preference pairs from paired successes and failures
  • Fine-tune an embedding model on trace-derived data and measure the retrieval quality lift in an agent’s semantic memory
  • Train a small language model from agent memory traces to replace a larger model on a scoped task, and evaluate the cost, latency, and quality trade-offs
  • Design the weight-space learning lifecycle: retrain triggers and cadence, checkpoint promotion gates, regression testing, and rollback

Course description

In this two-hour course, Richmond Alake teaches continual learning using training data most teams already own and ignore: their agents’ own memory traces. Every production agent records what was asked, retrieved, worked, and failed. That record is a fine-tuning dataset waiting to be curated, and because it holds successes and failures on the same tasks, it’s preference data too.

You’ll learn the two highest-leverage plays for improving your agent’s tokenomics: fine-tuning embedding models on trace-derived contrastive pairs, so retrieval sharpens on the queries your users actually ask, and training small language models from high-signal traces to replace larger models on scoped tasks, cutting cost and latency while measuring what quality you keep.

This live event is for you because...

  • You’re an ML engineer or AI engineer with agents in production generating interaction data you are not yet using.
  • You’re responsible for the cost and latency profile of an LLM-powered product and want smaller models doing more of the work.
  • You want practical fine-tuning grounded in agent systems rather than generic fine-tuning tutorials.

Prerequisites

  • A Python 3.11+ environment with PyTorch and the Hugging Face ecosystem installed (transformers, sentence-transformers, peft, datasets)
  • An API key for at least one frontier model provider (Anthropic, OpenAI, or equivalent)
  • A local database available for memory persistence (Oracle AI Database or bring your own database)
  • A code editor with notebook support, and Git installed for cloning the course repository shared before the session
  • Working knowledge of Python and comfort with model APIs
  • Familiarity with embeddings and retrieval concepts (vector search, semantic similarity)

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

When token space is not enough (45 minutes)

  • Presentation: The token-space versus weight-space decision—what belongs in context and memory, what belongs in the weights, the cost of getting it wrong, and where this course’s manual pipeline (curate traces, fine-tune, evaluate) sits relative to self-adapting models and sparse memory fine-tuning; from traces to training data—what agent memory captures, what makes a trace high-signal, the curation pipeline from raw trajectories to clean datasets, failures as ready-made preference pairs, and the verification caveat; parameter-efficient fine-tuning in practice (LoRA and adapters) and the honest forgetting story; how each of this course’s two plays handles that
  • Q&A
  • Break

Fine-tuning embedding models from memory traces (35 minutes)

  • Presentation: Why generic embeddings underperform on your domain; how trace-derived contrastive pairs encode what your users actually mean; deploying the updated embedding model behind the harness—re-indexing safely, versioning, and rollback
  • Hands-on exercise: Build a contrastive dataset from agent memory traces (positives from successful retrievals, hard negatives from retrieved-but-unused documents and failure traces); fine-tune an embedding model; evaluate twice—retrieval lift on held-out domain queries, and a regression check on general retrieval

Training small language models to replace larger models (40 minutes)

  • Presentation: Distilling scoped capability from traces; task selection; SFT dataset construction from successful trajectories; preference pairs from paired successes and failures; where a DPO pass fits; routing between small and large models in production; retraining triggers and cadence; checkpoint promotion gates and regression testing before a new model goes live; rollback; the economics of the replacement decision
  • Hands-on exercise: Train a small model on a trace-derived dataset, then run the three-way evaluation—the large model, the large model with token-space learning (a skill file from the companion course), and the fine-tuned SLM, scored on quality, cost, and latency
  • Q&A

Your Instructor

Richmond Alake

Richmond Alake is director of AI developer experience at Oracle, where he leads developer relations for the Oracle AI Database and pioneers the discipline of memory engineering, a term he coined to describe the structured design of agent memory systems. He runs the #100DaysOfAgentMemory LinkedIn series, which has reached tens of thousands of AI practitioners.

Richmond has published over 200 technical articles with more than one million views across Medium, NVIDIA’s technical blog, and other industry publications, covering topics from agent memory systems to production LLM application development. He has spoken at the AI Engineer World’s Fair, DeveloperWeek, Confluence, and Monki Gras. Previously, he served as an AI/ML developer advocate at MongoDB, where he built MemoRizz, an open source Python agent memory library, and codeveloped a DeepLearning.AI course on prompt compression and query optimization. Richmond holds a master’s degree in computer vision, machine learning, and robotics from the University of Surrey.

linkedinXlinksearch

Skill covered

QA / Testing