Hands-On Large Language Models

Hands-On Large Language Models

by Jay Alammar, Maarten Grootendorst
Released December 2024
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098150969

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

AI has acquired startling new language capabilities in just the past few years. Driven by the rapid advances in deep learning, language AI systems are able to write and understand text better than ever before. This trend enables the rise of new features, products, and entire industries. With this book, Python developers will learn the practical tools and concepts they need to use these capabilities today.

You'll learn how to use the power of pre-trained large language models for use cases like copywriting and summarization; create semantic search systems that go beyond keyword matching; build systems that classify and cluster text to enable scalable understanding of large amounts of text documents; and use existing libraries and pre-trained models for text classification, search, and clusterings.

This book also shows you how to:

  • Build advanced LLM pipelines to cluster text documents and explore the topics they belong to
  • Build semantic search engines that go beyond keyword search with methods like dense retrieval and rerankers
  • Learn various use cases where these models can provide value
  • Understand the architecture of underlying Transformer models like BERT and GPT
  • Get a deeper understanding of how LLMs are trained
  • Understanding how different methods of fine-tuning optimize LLMs for specific applications (generative model fine-tuning, contrastive fine-tuning, in-context learning, etc.)
  • Optimize LLMs for specific applications with methods such as generative model fine-tuning, contrastive fine-tuning, and in-context learning

Publisher resources

View/Submit Errata

Table of contents

  1. 1. Categorizing Text
    1. Supervised Text Classification
      1. Model Selection
      2. Data
      3. Classification Head
      4. Pre-Trained Embeddings
    2. Zero-shot Classification
      1. Pre-Trained Embeddings
      2. Natural Language Inference
    3. Classification with Generative Models
      1. In-Context Learning
      2. Named Entity Recognition
    4. Summary
  2. 2. Semantic Search
    1. Three Major Categories of Language-Model-based Search Systems
    2. Dense Retrieval
      1. Dense Retrieval Example
      2. Chunking Long Texts
      3. Nearest Neighbor Search vs. Vector Databases
      4. Fine-tuning embedding models for dense retrieval
    3. Reranking
      1. Reranking Example
      2. Open Source Retrieval and Reranking with Sentence Transformers
      3. How Reranking Models Work
    4. Generative Search
      1. What is Generative Search?
    5. Other LLM applications in search
      1. Evaluation metrics
    6. Summary
  3. 3. Text Clustering and Topic Modeling
    1. Text Clustering
      1. Data
      2. How do we perform Text Clustering?
    2. Topic Modeling
      1. BERTopic
      2. Example
      3. Representation Models
      4. Text Generation
      5. Topic Modeling Variations
    3. Summary
  4. 4. Tokens & Token Embeddings
    1. LLM Tokenization
      1. How tokenizers prepare the inputs to the language model
      2. Word vs. Subword vs. Character vs. Byte Tokens
      3. Comparing Trained LLM Tokenizers
      4. Tokenizer Properties
      5. A Language Model Holds Embeddings for the Vocabulary of its Tokenizer
      6. Creating Contextualized Word Embeddings with Language Models
    2. Word Embeddings
      1. Using Pre-trained Word Embeddings
      2. The Word2vec Algorithm and Contrastive Training
    3. Embeddings for Recommendation Systems
      1. Recommending songs by embeddings
    4. Summary
  5. About the Authors

Product information

  • Title: Hands-On Large Language Models
  • Author(s): Jay Alammar, Maarten Grootendorst
  • Release date: December 2024
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098150969