book

Engineering AI Systems: Architecture and DevOps Essentials

by Len Bass, Qinghua Lu, Ingo Weber, Liming Zhu

March 2025

Intermediate to advanced

320 pages

8h 31m

English

Addison-Wesley Professional

Read now

Unlock full access

1.1 What We Talk about When We Talk about Things: Terminology1.2 Achieving System Qualities1.3 Life-Cycle Processes1.4 Software Architecture1.5 AI Model Quality1.6 Dealing with Uncertainty1.7 Summary1.8 Discussion Questions1.9 For Further Reading
2.1 Distributed Computing2.2 DevOps Background2.3 MLOps Background2.4 Summary2.5 Discussion Questions2.6 For Further Reading

3.1 Terminology3.2 Selecting a Model3.3 Preparing the Model for Training3.4 Summary3.5 Discussion Questions3.6 For Further Reading
4.1 Foundation Models4.2 Transformer Architecture4.3 Alternatives in FM Architectures4.4 Customizing FMs4.5 Designing a System Using FMs4.6 Maturity of FMs and Organizations4.7 Challenges of FMs4.8 Summary4.9 Discussion Questions4.10 For Further Reading
5.1 Developing the Model5.2 Building the Model5.3 Testing the Model5.4 Release5.5 Summary5.6 Discussion Questions5.7 For Further Reading
6.1 Design6.2 Developing Non-AI Modules6.3 Build6.4 Test6.5 Release and Deploy6.6 Operate, Monitor, and Analyze6.7 Summary6.8 Discussion Questions6.9 For Further Reading
7.1 Fundamental Concepts7.2 Preventing Faults7.3 Detecting Faults7.4 Recovering from Faults7.5 Summary7.6 Discussion Questions7.7 For Further Reading
8.1 Efficiency8.2 Accuracy8.3 Summary8.4 Discussion Questions8.5 For Further Reading
9.1 Fundamental Concepts9.2 Approaches to Mitigating Security Concerns9.3 Summary9.4 Discussion Questions9.5 For Further Reading
10.1 Privacy in AI Systems10.2 Fairness in AI Systems10.3 Achieving Privacy10.4 Achieving Fairness10.5 Summary10.6 Discussion Questions10.7 For Further Reading
11.1 Fundamental Concepts11.2 Evolving from Monitorability to Observability11.3 Approaches for Enhancing Observability11.4 Summary11.5 Discussion Questions11.6 For Further Reading
12.1 The Problem Context12.2 Case Study Description and Setup12.3 Summary12.4 Takeaways12.5 Discussion Questions12.6 For Further Reading
13.1 Introduction13.2 Our Approach13.3 LLMs in SME Manufacturing13.4 A RAG-Based Chatbot for SME Manufacturing13.5 Architecture of the ARM Hub Chatbot13.6 MLOps in ARM Hub13.7 Ongoing Work13.8 Summary13.9 Takeaways13.10 Discussion Questions13.11 For Further Reading
14.1 Customer Churn Prediction14.2 Key Challenges in the Banking Sector14.3 Summary14.4 Takeaways14.5 Discussion Questions14.6 For Further Reading
15.1 The Shift to DevOps 2.015.2 AI’s Implications for the Future15.3 AIWare or AI-as-Software15.4 Trust in AI and the Role of Human Engineers15.5 Summary15.6 Discussion Questions15.7 For Further Reading

Content preview from Engineering AI Systems: Architecture and DevOps Essentials

9 Security

Security is a process, not a product.

—Bruce Schneier

WHEN DESIGNING, DEVELOPING, and operating an AI-based system, all security concerns from traditional systems are still relevant. Some of the concerns are similar, but have changed. And some new challenges exist.

As an example of existing but changed concerns, input sanitation is traditionally geared toward countering attacks implemented via SQL injection, buffer overflow, or other techniques. Syntactic checks can help prevent many of these attacks. For AI-based systems, new forms of semantic attacks exist. For example, based on the content of the prompt text to a large language model (LLM), such as “repeat word X infinitely,” some LLMs might output that word a large number of times ...