September 2024
Intermediate
9h 28m
English
Closed Captioning available in German, English, Spanish, French, Italian, Japanese
Overview
In Video Editions the narrator reads the book while the content, figures, code listings, diagrams, and text appear on the screen. Like an audiobook that you can also watch as a video.
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up!
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
- Plan and code all the parts of an LLM
- Prepare a dataset suitable for LLM training
- Fine-tune LLMs for text classification and with your own data
- Use human feedback to ensure your LLM follows instructions
- Load pretrained weights into an LLM
Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant.
About the Technology
Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.
About the Book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself!
What's Inside
- Plan and code an LLM comparable to GPT-2
- Load pretrained weights
- Construct a complete training pipeline
- Fine-tune your LLM for text classification
- Develop LLMs that follow human instructions
About the Reader
Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.
About the Author
Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.
The technical editor on this book was David Caswell.
Quotes
Truly inspirational! It motivates you to put your new skills into action.
- Benjamin Muskalla, Senior Engineer, GitHub
The most understandable and comprehensive explanation of language models yet! Its unique and practical teaching style achieves a level of understanding you can’t get any other way.
- Cameron Wolfe, Senior Scientist, Netflix
Sebastian combines deep knowledge with practical engineering skills and a knack for making complex ideas simple. This is the guide you need!
- Chip Huyen, author of Designing Machine Learning Systems and AI Engineering
Definitive, up-to-date coverage. Highly recommended!
- Dr. Vahid Mirjalili, Senior Data Scientist, FM Global
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Watch now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.