Skip to Content
View all events

Open Source Large Language Models in 3 Weeks

Published by O'Reilly Media, Inc.

Intermediate content levelIntermediate

Learn how to answer questions, use SBERT, Llama & Co and tailor them to your needs

Course outcomes:

  • Understand open source LLMs and how to modify existing LLMs with open source software
  • Be able to perform question answering with existing document corpora
  • Learn about advantages and disadvantages of existing LLMs
  • Be able to deploy LLMs in environments with limited resources using existing frontends

The (accidental) release of LLaMa, Meta’s large language model, has proven to be the big bang of open source language models. A lot of work has been invested by the community to fine-tune the model to specific needs like question answering or working as a chatbot. Additionally, the underlying training data has been analyzed in detail, and the community is collecting free training data. This has led to many new and more powerful, commercially usable open source LLMs. Join expert Christian Winkler to get a structured and consistent introduction to using open source LLMs. You’ll learn how to use models to find information with dense passage retrieval. You’ll get working, hands-on solutions and explanations on how these models can also excel on less powerful hardware by using new approaches to quantization. And you’ll also learn about different frontends these models can be plugged into. All code will be provided in the course and in GitHub.

NOTE: With today’s registration, you’ll be signed up for all three weeks. Although you can attend any of the sessions individually, we recommend participating in all three.

What you’ll learn and how you can apply it

  • Work with open source LLMs
  • Choose the correct base LLM
  • Generate text
  • Retrieve text (question answering)
  • Deploy in limited environment

This live event is for you because...

  • You’re a data scientist, ML engineer, or NLP developer.
  • You want to become an expert in large language models.
  • You want to use modern methods for business use cases.

Prerequisites

  • Set up a Google Colab account
  • For local installations, a powerful GPU is necessary)
  • Link to Jupyter Notebook
  • A working knowledge of Python and Jupyter notebooks
  • Machine learning and Hugging Face Transformers experience (helpful but not required)

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Week 1

Sentence embeddings (90 minutes)

  • Presentation: Introduction to similarity; finding semantically similar statements
  • Hands-on exercise: Jupyter notebook with SBERT
  • Group discussion: GPUs and hardware requirements
  • Q&A
  • Break

Cross encoders (90 minutes)

  • Presentation: Passage retrieval; improving prior solution
  • Hands-on exercises: Work with two-stage process for retrieving data; use existing software like txtai or langchain to achieve a similar result; work around scalability issues using a vector db; explore deployment options
  • Group discussion: scalability and hardware requirements
  • Q&A

Week 2

Generative LLMs (90 minutes)

  • Presentation: Introduction to transformers and open source language models and their differences (Llama, Phi-4, Qwen, DeepSeek); using existing models, converting them between different formats, and handling them with specific libraries or generic transformers
  • Hands-on exercises: Download and use existing model; convert to the correct format; use model-specific libraries for answering questions; convert to other formats
  • Group discussion: hardware, GPUs and deployment scenarios
  • Q&A
  • Break

Quantization and execution (90 minutes)

  • Presentation: Introduction to quantization; different ways of quantizing; GPTQ versus load_8bit
  • Hands-on exercises: Quantize an existing model; compare original results to quantized results
  • Group discussion: difference between PC and Apple hardware, GPUs
  • Q&A

Week 3

Deploying LLMs (90 minutes)

  • Presentation: Introduction and resource limits; reducing costs while retaining business value
  • Hands-on exercises: Compare results of smaller to large models; quantize using several GPUs and using expensive GPUs with lots of RAM
  • Group discussion: is it worth to buy your own GPU?
  • Q&A
  • Break

Frontend solutions (90 minutes)

  • Presentation: Introduction; focus on UX; different ready-made solutions like Open Web UI, Ollama, LLM studio
  • Hands-on exercise: Deploy different language models
  • Group discussion: Performance differences, avoiding double downloads
  • Q&A

Your Instructor

Christian Winkler

Christian Winkler has worked with large amounts of data for several decades, and specializes in advanced analytics. As a professor at the Technical University of Applied Science in Nuremberg, he concentrates his research on optimizing user experience using modern data-based approaches. He researches and publishes articles about data management and large language models and is a regular contributor to machine learning conferences.

Skill covered

Large Language Models (LLMs)