Skip to Content
View all events

Local LLMs Made Easy

Published by O'Reilly Media, Inc.

Beginner content levelBeginner

Using UI-Based Frameworks Jan AI and GPT4ALL

Course Outcomes:

  • Understand the fundamental concepts and benefits of (small) LLMs
  • Choose an LLM that can be run on your hardware
  • Get hands-on experience using two UI-based frameworks, Jan AI and GPT4all, to build and run a local LLM application
  • Understand and use Retrieval Augmented Generation (RAG) to feed your documents to the LLM

Discover the ease of running large language models (LLMs) on your machine using intuitive, user-friendly frameworks. In this hands-on course machine learning engineer Lisa Becker will introduce you to the world of LLMs without the need for extensive technical expertise. This course is perfect for beginners who want to get started with LLMs using simple, graphical interfaces.

You will learn to choose and implement small, efficient LLMs using popular UI-based frameworks such as Jan AI and GPT4all. Through guided exercises, you'll gain hands-on experience with setting up these tools, performing prompt engineering, and integrating custom data. By the end of the course, you'll be able to build and run your own local LLM applications, enhancing your projects with advanced AI capabilities in an accessible and straightforward manner.

What you’ll learn and how you can apply it

  • Choose the right LLM for your hardware and use case.
  • Use UI-based frameworks such as Jan AI and GPT4all.
  • Build custom, local LLM applications with Jan AI.
  • Develop LLM applications that access custom data (Retrieval Augmented Generation - RAG).

This live event is for you because...

  • Individuals who want to try out LLMs.
  • Those who do not have access to large compute resources or technical knowledge to run LLMs.

Prerequisites

  • Basic computer skills
  • Some familiarity with LLMs

Recommended preparation:

  • Install Jan AI and GPT4All (optional, to participate in exercises)
  • Have at least one document in PDF format (e.g., a resumé or CV) to use as a custom knowledge base for your LLM (optional)

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Introduction (5 minutes)

  • Presentation: Speaker and topic introduction, course overview and learning objectives

What are language models? (35 minutes)

  • Definition and context, explanation of what LLMs are (distinction between large LLMs and small LLMs), why use small LLMs? Benefits of small LLMs: efficiency, cost, privacy, and control…
  • Download and install Jan AI
  • Presentation: Overview and key features of smaller, open-source LLMs, such as Gemma (Google), Mistral (Mistral AI), Llama (Meta) (these are subject to change, depending on which LLMs are most popular at the time of the webinar being held)
  • Break (10 minutes)

Run LLMs with Jan AI (60 minutes)

  • Exercise: Find out what GPU you’re using
  • Find your way around Hugging Face to choose your LLM
  • Configure Jan AI
  • Work with your first LLM in Jan AI (introduction to LLM parameters and prompt engineering to summarize texts in different ways)
  • Download and install GPT4all for upcoming exercise
  • Break (10 minutes)

Run LLMs with GPT4all (45 minutes)

  • Exercise: Configure GPT4all
  • Set up previous or another LLM in GPT4all
  • Introduction to Retrieval-Augmented Generation (RAG) and how to add custom documents to GPT4all for the LLM to access
  • Text generation task including custom documents

Wrap-up (15 minutes)

  • Comprehensive overview of various applications and use cases of LLMs
  • Discuss challenges and limitations of using small LLMs
  • Q&A

Your Instructor

  • Lisa Becker

    Lisa Becker is a machine learning engineer specializing in natural language processing and large language models at Pure App. With a master’s degree in cognitive systems from the University of Potsdam, Lisa applies her expertise to domains that include speech recognition, sentiment analysis, and text generation, leveraging her technical and linguistic skills to enhance the user experience and engagement of Pure App. She develops and deploys NLP and LLM solutions that enable users to communicate more effectively, discover compatible matches, and receive personalized recommendations. She also contributes to the research and innovation of the app, exploring new ways of using AI to create meaningful and authentic connections. Lisa is a passionate advocate for diversity, equity, and inclusion and is dedicated to sharing her knowledge as a public speaker, blogger, and course instructor for O’Reilly. As a certified #IAmRemarkable facilitator, she empowers individuals to celebrate their achievements and break through societal biases.

Skills covered

  • Large Language Models (LLMs)
  • Generative AI
  • Unified Modeling Language (UML)