book

Build Applications with Local AI Models on a Mac

Name: Build Applications with Local AI Models on a Mac
Author: Yoshiki Kanda
ISBN: 9781633433700

by Yoshiki Kanda

October 2026

296 pages

7h 34m

English

Manning Publications

Read now

Unlock full access

Build Applications with Local AI Models on a Mac
copyright
contents
preface
acknowledgments
about this book
about the author
about the cover illustration
Part 1 First steps: Entering the world of the terminal
1 Getting started with local AI
1.1 Why run AI locally?1.1.1 Cloud AI vs. local AI1.1.2 When to use cloud AI vs. local AI1.2 The local chat application you will build1.3 What you’ll need1.4 What is a large language model?1.5 Using the terminal to interact with your computer1.5.1 Two ways to talk to your computer1.5.2 CLI vs. GUI: A comparison1.5.3 Why use the command line?1.6 Opening the terminal on macOS1.6.1 Method 1: Spotlight search (recommended)1.6.2 Method 2: Applications folder1.6.3 Customizing the appearance1.7 Essential commands1.7.1 Where am I?1.7.2 What is here?1.7.3 Moving around1.7.4 Creating folders1.7.5 What happens when you type a command1.7.6 Common fears and realities

2 Installing and using Homebrew
2.1 Package managers: App stores for developers2.2 Installing Homebrew2.2.1 Understanding the command2.2.2 Running the installation2.2.3 Post-installation setup2.3 Verifying the installation2.4 Installing your first package: Wget2.4.1 Installing wget2.4.2 Trying out wget2.5 Key Homebrew commands2.5.1 Installing a package2.5.2 Searching for a package2.5.3 Listing installed packages2.5.4 Getting package information2.5.5 Updating Homebrew2.5.6 Upgrading installed packages2.5.7 Uninstalling a package2.5.8 Checking for problems2.6 How Homebrew organizes files
3 Installing and setting up Ollama
3.1 What is Ollama?3.1.1 Why Ollama?3.1.2 How Ollama works3.2 Cloud AI vs. local AI with Ollama3.3 Installing Ollama3.3.1 Step 1: Update Homebrew3.3.2 Step 2: Install Ollama3.3.3 Step 3: Verify the installation3.4 Starting the Ollama service3.4.1 Starting the service3.4.2 Verifying the service3.4.3 What happens at startup3.4.4 Stopping the service3.4.5 Troubleshooting common problems
Part 2 Core skills: Building your first AI chatbot
4 Downloading an LLM and having your first conversation
4.1 Downloading your first model4.1.1 Choosing a starting model4.1.2 Running the pull command4.1.3 Understanding the model name format4.2 Interactive chat4.2.1 Starting interactive chat mode4.2.2 A sample conversation4.2.3 Exiting chat mode4.3 Understanding model sizes and parameters4.3.1 What are parameters?4.3.2 The size, capability, and memory tradeoff4.3.3 What “quantization” means4.4 Model comparison: Choosing the right model4.4.1 The major model families4.4.2 Downloading a second model4.4.3 A quick model comparison test4.5 Model management commands4.5.1 Listing downloaded models4.5.2 Checking running models4.5.3 Viewing model details4.5.4 Removing a model
5 Setting up VS Code and your Python development environment
5.1 What is VS Code?5.2 Installing VS Code5.2.1 Step 1: Install with Homebrew5.2.2 Step 2: Verify the installation5.2.3 Step 3: Open VS Code5.3 Installing Python 3.135.4 Installing the Python extension5.4.1 Installation steps5.4.2 What you will see in practice5.5 Using the integrated terminal5.5.1 Opening the terminal5.5.2 Verifying your tools5.6 Creating your project folder5.6.1 Setting up the workspace5.6.2 Creating and running your first file5.6.3 Project structure preview
6 Creating and managing Python virtual environments
6.1 Why virtual environments matter6.2 Creating a virtual environment6.2.1 Navigate to your project folder6.2.2 Create the virtual environment6.2.3 Understanding the venv command6.2.4 What is inside the venv folder?6.3 Activating and deactivating a virtual environment6.3.1 Activating the virtual environment6.3.2 Verifying activation6.3.3 Deactivating the virtual environment6.4 Installing packages with pip6.4.1 What is pip?6.4.2 Installing the Ollama Python package6.4.3 Installing Streamlit6.4.4 Checking the installed packages6.5 Freezing and restoring dependencies6.5.1 Creating the requirements.txt file6.5.2 Restoring an environment from requirements.txt6.5.3 Keeping requirements.txt updated6.5.4 The complete virtual environment workflow
7 Controlling LLMs with the Ollama Python library
7.1 Your first Ollama Python script7.1.1 The simplest AI program7.1.2 Breaking down the code7.2 Understanding the message format7.2.1 The three LLM message roles7.2.2 Adding a system message7.2.3 Experimenting with different system messages7.3 Streaming responses for real-time output7.3.1 The problem with waiting for the model’s response7.3.2 Implementing streaming7.3.3 Understanding the code7.3.4 The streaming communication flow7.4 Maintaining conversation history7.4.1 The LLM’s memory problem7.4.2 Building a conversation loop7.4.3 Running the conversation7.4.4 How the history list grows7.4.5 The complete conversation flow
8 Building a web UI with Streamlit
8.1 What Is Streamlit?8.2 Your first Streamlit app8.2.1 Step 1: Install Streamlit8.2.2 Step 2: Create the app file8.2.3 Step 3: Run the app8.2.4 Step 4: Add interactivity8.2.5 How Streamlit’s execution model works8.3 Core Streamlit components8.3.1 Text display8.3.2 User input8.3.3 Buttons and actions8.3.4 The sidebar8.3.5 Column layout8.4 Running Streamlit apps8.4.1 Starting and stopping8.4.2 Auto-reload8.4.3 Ports and multiple apps8.4.4 The development workflow8.4.5 Helpful command-line options8.5 Putting it all together
9 Recording audio and transcribing speech with MLX Whisper
9.1 What is speech recognition?9.2 Why use MLX Whisper on Apple silicon?9.3 Installing MLX Whisper9.4 Understanding digital audio9.4.1 Audio sample rate9.4.2 Audio channels9.4.3 Sample bit depth and data type9.5 Recording from the microphone9.5.1 sd.rec(): The recording call9.5.2 Squeezing the channel dimension9.5.3 Converting to int16 for WAV9.6 Your first transcription9.7 Choosing the right Whisper model9.7.1 Which model should you use?9.7.2 Trying a larger model9.8 Building a reusable transcription script9.9 Troubleshooting9.9.1 “No default input device”9.9.2 Audio is captured but Whisper returns empty text9.9.3 “Cannot import mlx_whisper”9.9.4 The first download is slow9.9.5 “File not found” for ffmpeg
10 Building a voice-enabled AI chat application
10.1 The voice AI pipeline10.2 Setting up the voice chat application project10.3 Building the application step by step10.3.1 Stage 1: Transcription in Streamlit10.3.2 Why use a temporary WAV file?10.3.3 Stage 2: Connecting transcription to the LLM10.3.4 Stage 3: The complete application10.4 The complete voice chat application10.4.1 Running the application10.4.2 Updating the Ollama model menu10.5 Understanding the complete voice chat application code10.5.1 Browser page configuration10.5.2 The sidebar: Controlling two models10.5.3 transcribe_audio(): The bridge between Streamlit and MLX Whisper10.5.4 stream_response(): A reusable streaming function10.5.5 handle_user_message(): The unified entry point10.5.6 Dual input: Voice and text10.6 Running and testing the application10.6.1 Startup checklist10.6.2 Testing checklist10.6.3 Troubleshooting10.7 Understanding the latency budget
11 Managing session state and chat history
11.1 The problem: LLMs are stateless11.2 Understanding Streamlit session state11.2.1 The script rerun problem11.2.2 Session state in action11.2.3 What you can store in session state11.2.4 When session state resets11.3 How LLMs use conversation history11.4 Context window management11.4.1 What happens when you exceed the context window11.4.2 Implementing history trimming11.4.3 The limited history tradeoff11.5 Adding a reset button11.6 Other useful session state patterns11.7 The complete chat application data flow
Part 3 Advanced topics: Deeper understanding and customization
12 Comparing and selecting LLM models
12.1 Model families and their strengths12.1.1 Llama (Meta)12.1.2 Gemma (Google)12.1.3 Qwen (Alibaba Cloud)12.1.4 Mistral (Mistral AI)12.1.5 DeepSeek (DeepSeek AI)12.1.6 The model ecosystem12.2 Model size vs. quality tradeoffs12.2.1 Model comparison quick reference12.2.2 Understanding tokens per second12.2.3 Benchmarks: What they tell you (and what they don’t)12.2.4 The RAM reality check12.3 Multilingual capabilities12.3.1 Language strengths by model family12.3.2 A side-by-side experiment12.3.3 Multilingual recommendation summary12.4 Switching models in your chatbot12.4.1 Step 1: Download the new model12.4.2 Step 2: Change one line of chat application code12.4.3 Step 3: Add a Model Selector in the Streamlit UI12.4.4 Managing your model collection12.4.5 A quick model comparison script12.5 The 2026 revolution: Frontier open models12.5.1 The mixture of experts revolution12.5.2 Model 1: Zhipu GLM-512.5.3 Model 2: Alibaba Qwen3-Coder-Next12.5.4 Model 3: MiniMax M2.512.5.5 The February 2026 benchmark landscape12.5.6 The new model comparison table12.5.7 What this means for you
13 System prompts and parameter tuning
13.1 System prompts explained13.2 Example system prompts: Personality presets13.2.1 Preset 1: The professional consultant13.2.2 Preset 2: The patient study buddy13.2.3 Preset 3: The pirate tutor13.2.4 Preset 4: The strict coding mentor13.2.5 Preset 5: The concise expert13.3 LLM temperature and creative parameters13.3.1 What temperature does13.3.2 Other important parameters13.4 Creating custom models with Ollama Modelfiles13.4.1 The Modelfile format13.4.2 Key Modelfile instructions13.4.3 Building a custom model step by step13.4.4 Managing custom models13.5 Adding customization controls to the Streamlit UI13.5.1 The enhanced chatbot13.5.2 How the system prompt affects the pipeline
14 Offline LLMs and their benefits
14.1 Verifying offline chatbot operation: The airplane mode test14.2 AI privacy and security analysis14.2.1 Cloud AI data flow14.2.2 Local AI data flow14.2.3 Why privacy matters: Concrete scenarios14.3 AI cost analysis14.3.1 Cloud AI costs14.3.2 Local AI costs14.3.3 The hidden cost: Dependency14.3.4 When cloud AI is worth the cost14.3.5 The long-term perspective on costs14.4 Use cases where local AI excels14.4.1 Healthcare14.4.2 Legal practice14.4.3 Education14.4.4 Creative work14.4.5 Journalism and research14.4.6 Personal finance and mental health14.4.7 Enterprise and corporate use14.4.8 Offline and remote environments14.4.9 Use case decision framework
15 Common errors and troubleshooting
15.1 Common errors and solutions15.1.1 Error: Model not found15.1.2 Error: Connection refused15.1.3 Error: Out of memory15.1.4 Error: Port already in use15.1.5 Error: Streamlit ModuleNotFoundError15.1.6 Error: Python version problems15.2 Reading error messages effectively15.3 Debugging techniques15.3.1 Using try/except blocks15.3.2 Step-by-step verification15.3.3 Print debugging15.3.4 Logging15.4 The troubleshooting roadmap15.4.1 Four levels of troubleshooting15.4.2 A note on asking for help
16 Advancing to RAG, Web UI, and fine-tuning
16.1 RAG: Retrieval-augmented generation16.1.1 What is RAG?16.1.2 How vectors and embeddings work16.1.3 A simple RAG implementation16.1.4 Loading real documents for RAG16.2 Using Open WebUI for a ChatGPT-like experience16.2.1 Using Open WebUI with Docker16.2.2 Key features of Open WebUI16.3 Custom model creation with Modelfiles16.3.1 Creating a Modelfile16.3.2 Advanced Modelfile options16.4 LangChain integration16.4.1 Installing LangChain16.4.2 Basic LangChain with Ollama16.4.3 Creating a chain16.4.4 Why LangChain matters16.5 Agentic AI patterns16.5.1 A simple agent concept16.5.2 The agent pattern: Evaluate and act16.5.3 Real-world agent use cases16.5.4 OpenClaw: The autonomous agent that drew major GitHub attention16.5.5 PicoClaw: An agent that fits in your pocket16.5.6 The API cost revolution: Why open models matter for agents16.6 Fine-tuning overview16.6.1 When to fine-tune (and when not to)16.6.2 The GGUF format for quantized models16.6.3 Fine-tuning workflow (high level)16.6.4 Unsloth: The fine-tuning accelerator16.6.5 GGUF quantization deep dive16.7 Project ideas and learning resources16.7.1 Ten project ideas16.7.2 Learning resources
17 The open model revolution of 2026
17.1 February 2026: A turning point17.2 GLM-517.2.1 Architecture17.2.2 Trained on Huawei chips17.2.3 Benchmark results17.2.4 License17.3 Qwen3-Coder-Next17.3.1 Ultra-Sparse MoE architecture17.3.2 Coding agent focus17.4 MiniMax M2.517.4.1 SWE-bench parity17.4.2 Cost advantage17.5 The benchmark picture17.6 The economics of open models17.6.1 Agent cost analysis17.6.2 Why local agents change what is possible17.7 The Unsloth-to-Ollama pipeline17.7.1 The complete custom model workflow17.7.2 Why customization matters now17.8 What this means for you17.8.1 How to try these models17.8.2 Practical next steps17.9 Further learning17.10 Beyond individual models: The rise of orchestration
18 Where to go from here
18.1 What you have accomplished18.2 Your learning roadmap18.2.1 Level 1: Foundation (what this book covered)18.2.2 Level 2: Intermediate (the next 3–6 months)18.2.3 Level 3: Advanced (6–12+ months)18.3 Final thoughts
appendix A Command reference
A.1 Terminal (macOS/Linux) commandsA.2 Homebrew commandsA.3 Ollama commandsA.4 Python and pip commandsA.5 Streamlit commandsA.6 Source code repository for this bookA.7 Git commandsA.8 Quick reference card
appendix B References and additional resources
B.1 ReferencesB.1.1 Chapter 4B.1.2 Chapter 5B.1.3 Chapter 17B.2 Additional resourcesB.2.1 Academic papers and technical reportsB.2.2 Model releases and technical reportsB.2.3 BooksB.2.4 Software, frameworks, and documentationB.2.5 Regulatory and policy frameworksB.2.6 Community resources

Content preview from Build Applications with Local AI Models on a Mac

10 Building a voice-enabled AI chat application

This chapter covers

The architecture of a voice AI pipeline
Connecting MLX Whisper to Streamlit
Refactoring the app into functions
Adding a text fallback so the app also works with keyboard input
Testing the full voice conversation loop
Understanding what makes this application challenging and how to extend it

This is the chapter where everything comes together. In the previous chapters, you learned how to use the terminal, install Ollama, pull AI models, write Python, call the Ollama API, build web interfaces with Streamlit, and transcribe speech with MLX Whisper. In this chapter, you will combine all of those skills into a voice-enabled AI chat application: speak into your microphone, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Finding Applications for Technologies Beyond the Core Business

Publisher Resources

ISBN: 9781633433700Publisher Support Publisher Website Purchase Link Other

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Build Applications with Local AI Models on a Mac

by Yoshiki Kanda