book

Domain-Specific Small Language Models

Name: Domain-Specific Small Language Models
Author: Guglielmo Iozzia
ISBN: 9781633436701

by Guglielmo Iozzia

May 2026

Intermediate

376 pages

9h 52m

English

Manning Publications

Read now

Unlock full access

Domain-Specific Small Language Models
copyright
contents
foreword
preface
acknowledgments
about this book
about the author
about the cover illustration
Part 1 First steps

1 Small language models
1.1 What are small language models?1.2 The Transformer architecture1.3 Areas of application1.4 The open source revolution1.5 Risks and challenges with generalist LLMs1.6 Domain-specific vs. generalist LLMs for business value
Part 2 Core domain-specific LLMs
2 Tuning for a specific domain
2.1 Data preparation2.1.1 Data preparation for BERT fine-tuning2.1.2 Data preparation for GPT fine-tuning2.1.3 Data preparation for RAG2.2 Retrieval-augmented generation2.3 Fine-tuning2.4 LoRA2.5 RAG or fine-tuning?
3 End-to-end transformer fine-tuning
3.1 Data preparation3.2 Fine-tuning3.3 Testing the fine-tuned model3.4 Domain-specific evaluation
4 Running inference
4.1 How to generate content4.1.1 Text completion4.1.2 Few-shot learning4.1.3 Code generation4.1.4 Evaluating the generated content4.2 Calculating inference cost4.3 Areas for improvement (cost savings and performance)4.3.1 Getting the most from your GPU4.3.2 Batching4.3.3 Estimating the generation time4.3.4 Optimizing GPU use with DeepSpeed
5 Exploring ONNX
5.1 The ONNX format5.2 ONNX operators and types5.3 The ONNX runtime5.4 ONNX runtime providers5.5 ONNX for LLMs on CPU5.6 ONNX for LLMs on GPU5.6.1 ONNX for GPT on GPU5.6.2 I/O binding
6 Quantizing for your production environment
6.1 Transformers precision formats6.2 8-bit quantization6.2.1 Hands-on 8-bit quantization6.2.2 LLM.int8() and quantization6.3 8-bit quantization with ONNX6.4 4-bit quantization6.4.1 4-bit quantization with GPTQ6.4.2 4-bit quantization with ggml
Part 3 Real-world use cases
7 Generating Python code
7.1 Using Transformers to generate code7.2 Generating Python code with a Transformer architecture7.2.1 Python code generation with CodeGen7.2.2 Using ONNX with models not supported by Optimum7.2.3 Model evaluation7.2.4 Python code generation with better models7.3 Coding assistance on commodity hardware
8 Generating protein structures
8.1 Applying Transformers in chemistry8.2 From natural language to protein structures8.3 Antibody generation with an SLM8.4 From CIF files to crystal structures
Part 4 Advanced concepts
9 Advanced quantization techniques
9.1 FlexGen9.2 SmoothQuant9.3 BitNet9.4 BitNet and Python
10 Profiling insights
10.1 Profiling ONNX-ported LLMs10.2 Transforming raw ONNX profiling data into insights10.3 Optimization of ONNX graphs for LLMs
11 Deployment and serving
11.1 vLLM11.1.1 Offline serving11.1.2 Online serving11.2 FastAPI11.2.1 Benchmarking various models11.2.2 Deploying the best-performing model with FastAPI11.3 MLC LLM11.4 Deployment and inference on Android devices11.4.1 MLC LLM framework11.4.2 MLLM framework11.4.3 Hugging Face Transformers
12 Running on your laptop
12.1 Why use a personal local assistant12.2 Running an LLM locally with Ollama12.2.1 Importing a custom model into Ollama12.2.2 User privacy in Ollama12.3 Running an LLM locally with LM Studio12.4 The LM Studio Python SDK12.5 Running an LLM locally with Jan12.6 The Cortex local LLM engine
13 Creating end-to-end LLM applications
13.1 Why LLMs alone aren’t enough13.2 Combining a domain-specific SLM with RAG13.3 Using a vector database13.4 Building an agent
14 Advanced components for LLM applications
14.1 GraphRAG14.1.1 Microsoft’s open source GraphRAG capabilities14.1.2 Evaluation metrics14.2 RAG + Agentic AI14.3 Long- and short-term memory management
15 Test-time compute and small language models
15.1 Test-time compute15.2 The OptiLLM inference proxy15.3 SLMs with embedded test-time compute15.4 Building a reasoning domain-specific SLM

Content preview from Domain-Specific Small Language Models

6 Quantizing for your production environment

This chapter covers

Precision formats for training and running LLMs
Quantizing LLMs to smaller-precision formats
Quantizing LLMs with multiple techniques and libraries

In chapter 5, you learned the core ONNX concepts and capabilities. I only briefly mentioned model quantization, but it’s crucial for LLM inference performance, so it deserves its own chapter. That’s our focus here.

6.1 Transformers precision formats

Previous chapters have mentioned several numeric precision formats used for LLM training and inference. It’s time to look at them more closely and to introduce additional formats used for model quantization.

In traditional scientific computing, 64-bit floating point (double ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633436701Publisher Support Other Publisher Website Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Domain-Specific Small Language Models

by Guglielmo Iozzia