book

Domain-Specific Small Language Models

Name: Domain-Specific Small Language Models
Author: Guglielmo Iozzia
ISBN: 9781633436701

by Guglielmo Iozzia

May 2026

Intermediate

376 pages

9h 52m

English

Manning Publications

Read now

Unlock full access

Domain-Specific Small Language Models
copyright
contents
foreword
preface
acknowledgments
about this book
about the author
about the cover illustration
Part 1 First steps

1 Small language models
1.1 What are small language models?1.2 The Transformer architecture1.3 Areas of application1.4 The open source revolution1.5 Risks and challenges with generalist LLMs1.6 Domain-specific vs. generalist LLMs for business value
Part 2 Core domain-specific LLMs
2 Tuning for a specific domain
2.1 Data preparation2.1.1 Data preparation for BERT fine-tuning2.1.2 Data preparation for GPT fine-tuning2.1.3 Data preparation for RAG2.2 Retrieval-augmented generation2.3 Fine-tuning2.4 LoRA2.5 RAG or fine-tuning?
3 End-to-end transformer fine-tuning
3.1 Data preparation3.2 Fine-tuning3.3 Testing the fine-tuned model3.4 Domain-specific evaluation
4 Running inference
4.1 How to generate content4.1.1 Text completion4.1.2 Few-shot learning4.1.3 Code generation4.1.4 Evaluating the generated content4.2 Calculating inference cost4.3 Areas for improvement (cost savings and performance)4.3.1 Getting the most from your GPU4.3.2 Batching4.3.3 Estimating the generation time4.3.4 Optimizing GPU use with DeepSpeed
5 Exploring ONNX
5.1 The ONNX format5.2 ONNX operators and types5.3 The ONNX runtime5.4 ONNX runtime providers5.5 ONNX for LLMs on CPU5.6 ONNX for LLMs on GPU5.6.1 ONNX for GPT on GPU5.6.2 I/O binding
6 Quantizing for your production environment
6.1 Transformers precision formats6.2 8-bit quantization6.2.1 Hands-on 8-bit quantization6.2.2 LLM.int8() and quantization6.3 8-bit quantization with ONNX6.4 4-bit quantization6.4.1 4-bit quantization with GPTQ6.4.2 4-bit quantization with ggml
Part 3 Real-world use cases
7 Generating Python code
7.1 Using Transformers to generate code7.2 Generating Python code with a Transformer architecture7.2.1 Python code generation with CodeGen7.2.2 Using ONNX with models not supported by Optimum7.2.3 Model evaluation7.2.4 Python code generation with better models7.3 Coding assistance on commodity hardware
8 Generating protein structures
8.1 Applying Transformers in chemistry8.2 From natural language to protein structures8.3 Antibody generation with an SLM8.4 From CIF files to crystal structures
Part 4 Advanced concepts
9 Advanced quantization techniques
9.1 FlexGen9.2 SmoothQuant9.3 BitNet9.4 BitNet and Python
10 Profiling insights
10.1 Profiling ONNX-ported LLMs10.2 Transforming raw ONNX profiling data into insights10.3 Optimization of ONNX graphs for LLMs
11 Deployment and serving
11.1 vLLM11.1.1 Offline serving11.1.2 Online serving11.2 FastAPI11.2.1 Benchmarking various models11.2.2 Deploying the best-performing model with FastAPI11.3 MLC LLM11.4 Deployment and inference on Android devices11.4.1 MLC LLM framework11.4.2 MLLM framework11.4.3 Hugging Face Transformers
12 Running on your laptop
12.1 Why use a personal local assistant12.2 Running an LLM locally with Ollama12.2.1 Importing a custom model into Ollama12.2.2 User privacy in Ollama12.3 Running an LLM locally with LM Studio12.4 The LM Studio Python SDK12.5 Running an LLM locally with Jan12.6 The Cortex local LLM engine
13 Creating end-to-end LLM applications
13.1 Why LLMs alone aren’t enough13.2 Combining a domain-specific SLM with RAG13.3 Using a vector database13.4 Building an agent
14 Advanced components for LLM applications
14.1 GraphRAG14.1.1 Microsoft’s open source GraphRAG capabilities14.1.2 Evaluation metrics14.2 RAG + Agentic AI14.3 Long- and short-term memory management
15 Test-time compute and small language models
15.1 Test-time compute15.2 The OptiLLM inference proxy15.3 SLMs with embedded test-time compute15.4 Building a reasoning domain-specific SLM

Content preview from Domain-Specific Small Language Models

4 Running inference

This chapter covers

Generating different types of content
Calculating the cost of inference with a large model
Identifying areas for performance improvements and cost savings

Chapters 2 and 3 presented examples of data preparation and SLM tuning. Now I’ll introduce SLM inference and offer tips for estimating GPU costs and finding performance and cost improvements.

4.1 How to generate content

In this chapter, we’ll use a language model as a reference for content generation. We’ll cover the key factors that drive execution speed, accuracy, and compute cost—background that you’ll use when applying the diverse quantization methods I’ll explain in chapters 5, 6, and 9.

Our reference open source model is GPT-Neo large ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633436701Publisher Support Other Publisher Website Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Domain-Specific Small Language Models

by Guglielmo Iozzia