book

Engineering AI Systems: Architecture and DevOps Essentials

by Len Bass, Qinghua Lu, Ingo Weber, Liming Zhu

March 2025

Intermediate to advanced

320 pages

8h 31m

English

Addison-Wesley Professional

Read now

Unlock full access

1.1 What We Talk about When We Talk about Things: Terminology1.2 Achieving System Qualities1.3 Life-Cycle Processes1.4 Software Architecture1.5 AI Model Quality1.6 Dealing with Uncertainty1.7 Summary1.8 Discussion Questions1.9 For Further Reading
2.1 Distributed Computing2.2 DevOps Background2.3 MLOps Background2.4 Summary2.5 Discussion Questions2.6 For Further Reading

3.1 Terminology3.2 Selecting a Model3.3 Preparing the Model for Training3.4 Summary3.5 Discussion Questions3.6 For Further Reading
4.1 Foundation Models4.2 Transformer Architecture4.3 Alternatives in FM Architectures4.4 Customizing FMs4.5 Designing a System Using FMs4.6 Maturity of FMs and Organizations4.7 Challenges of FMs4.8 Summary4.9 Discussion Questions4.10 For Further Reading
5.1 Developing the Model5.2 Building the Model5.3 Testing the Model5.4 Release5.5 Summary5.6 Discussion Questions5.7 For Further Reading
6.1 Design6.2 Developing Non-AI Modules6.3 Build6.4 Test6.5 Release and Deploy6.6 Operate, Monitor, and Analyze6.7 Summary6.8 Discussion Questions6.9 For Further Reading
7.1 Fundamental Concepts7.2 Preventing Faults7.3 Detecting Faults7.4 Recovering from Faults7.5 Summary7.6 Discussion Questions7.7 For Further Reading
8.1 Efficiency8.2 Accuracy8.3 Summary8.4 Discussion Questions8.5 For Further Reading
9.1 Fundamental Concepts9.2 Approaches to Mitigating Security Concerns9.3 Summary9.4 Discussion Questions9.5 For Further Reading
10.1 Privacy in AI Systems10.2 Fairness in AI Systems10.3 Achieving Privacy10.4 Achieving Fairness10.5 Summary10.6 Discussion Questions10.7 For Further Reading
11.1 Fundamental Concepts11.2 Evolving from Monitorability to Observability11.3 Approaches for Enhancing Observability11.4 Summary11.5 Discussion Questions11.6 For Further Reading
12.1 The Problem Context12.2 Case Study Description and Setup12.3 Summary12.4 Takeaways12.5 Discussion Questions12.6 For Further Reading
13.1 Introduction13.2 Our Approach13.3 LLMs in SME Manufacturing13.4 A RAG-Based Chatbot for SME Manufacturing13.5 Architecture of the ARM Hub Chatbot13.6 MLOps in ARM Hub13.7 Ongoing Work13.8 Summary13.9 Takeaways13.10 Discussion Questions13.11 For Further Reading
14.1 Customer Churn Prediction14.2 Key Challenges in the Banking Sector14.3 Summary14.4 Takeaways14.5 Discussion Questions14.6 For Further Reading
15.1 The Shift to DevOps 2.015.2 AI’s Implications for the Future15.3 AIWare or AI-as-Software15.4 Trust in AI and the Role of Human Engineers15.5 Summary15.6 Discussion Questions15.7 For Further Reading

Content preview from Engineering AI Systems: Architecture and DevOps Essentials

2 Software Engineering Background

You can’t build a great building on a weak foundation. You must have a solid foundation if you’re going to have a strong superstructure.

—Nelson Rockefeller

BEFORE DIVING INTO AI engineering, it is necessary to have a solid background in various aspects of distributed computing, DevOps, and MLOps. We provide this background in this chapter. Feel free to skip or skim the parts with which you are familiar, but note that we often connect the background information with its implications for AI systems. We cover the following topics in this chapter: