9 GPT-ing on the go
This chapter covers
- Running a large language model locally
- Comparing the results of two locally hosted large language models against those of ChatGPT
- Determining when using offline models is appropriate
Imagine you are on your way to an AI conference halfway around the world. You are on a plane, cruising at 35,000 feet above the ground, and you want to prototype a new feature for your application. The airplane’s Wi-Fi is prohibitively slow and expensive. What if instead of paying all that money for a broken and borderline unusable GPT, you have one running right there on your laptop, offline? This chapter will review developers’ options to run a large language model (LLM) locally.
9.1 Motivating theory
The introductory scenario ...
Get AI-Powered Developer now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
