book

Conversational AI

Name: Conversational AI
Author: Andrew Freed
ISBN: 9781617298837

by Andrew Freed

October 2021

Intermediate to advanced

320 pages

9h 21m

English

Manning Publications

Audiobook available

Read now

Unlock full access

inside front cover
Conversational AI
Copyright
dedication
brief contents
contents
front matter
prefaceacknowledgmentsabout this bookWho should read this bookHow this book is organized: A roadmapAbout the codeliveBook discussion forumOther online resourcesabout the authorabout the cover illustration
Part 1. Foundations
1 Introduction to conversational AI
1.1 Introduction to AI assistants and their platforms1.1.1 Types of AI assistants1.1.2 A snapshot of AI assistant platforms1.2 Primary use cases for AI assistant technology1.2.1 Self-service assistant1.2.2 Agent assist1.2.3 Classification and routing1.3 Follow along with this book1.3.1 What you need to create your assistant1.3.2 Useful spreadsheet software1.3.3 Recommended programming language and code repositorySummary
2 Building your first conversational AI
2.1 Building a conversational AI for Fictitious Inc.2.2 What’s the user’s intent?2.2.1 What’s an utterance?2.2.2 What’s a response?2.2.3 How does the assistant understand what the user means?2.2.4 Why machine learning?2.2.5 What’s an intent?2.2.6 What’s an entity?2.2.7 Combining intents and entities2.2.8 intents to the Fictitious Inc. assistant2.3 Responding to the user2.3.1 Simple question-and-answer responses2.3.2 Contextualizing a response by using entities2.3.3 An alternative way to provide contextual responses2.3.4 Responding with a process flow2.4 Other useful responses2.4.1 Detecting low confidence2.4.2 Counting misunderstandings2.4.3 Implementing confidence detection and the two-strikes rule2.5 Try to build this conversational AI yourselfSummary

Part 2. Designing for success
3 Designing effective processes
3.1 What processes will the assistant handle?3.1.1 Designing for the most common user needs3.1.2 Assembling a plan and a dream team3.1.3 Managing the design process3.1.4 Cross-cutting design aspects3.2 Choosing the channel to implement first3.2.1 How users receive information in voice and web3.2.2 How the assistant receives information in voice and webSummary
4 Designing effective dialogue
4.1 Writing dialogue4.1.1 Take a conversational tone4.1.2 Don’t repeat yourself (much)4.1.3 Acknowledge the user4.2 Asking questions4.3 What if the assistant doesn’t understand?4.3.1 Reprompting4.3.2 Disambiguation4.3.3 EscalationSummary
5 Building a successful AI assistant
5.1 AI assistant use cases5.2 Conversational AI success metrics5.2.1 Containment5.2.2 Time to resolution5.2.3 Net promoter score5.2.4 Coverage5.2.5 Instrumenting your conversational AI5.3 Command interpreter success metrics5.3.1 Usage5.3.2 Stickiness5.3.3 Instrumenting your command interpreter5.4 Event classifier success metrics5.4.1 Time to resolution5.4.2 Number of hand-offs5.4.3 Other customer satisfaction metrics5.4.4 Instrumenting your classifierSummary
Part 3. Training and testing
6 Training your assistant
6.1 Training an AI assistant6.2 Finding training data6.2.1 Production logs6.2.2 A mock user interface6.2.3 Subject-matter experts6.2.4 Organizing training data into intents6.3 Assessing whether you have the right training data6.3.1 Training data variety6.3.2 Training data volume6.3.3 Training data veracitySummary
7 How accurate is your assistant?
7.1 Testing an AI assistant for accuracy7.1.1 Testing a single utterance7.1.2 Testing multiple utterances7.1.3 Selecting a test data set7.2 Comparing testing methodologies7.2.1 Blind testing7.2.2 k-folds cross-validation test7.3 Selecting the right accuracy metric for the jobSummary
8 Testing your dialogue flows
8.1 Functionally testing a dialogue flow8.1.1 Manually testing a conversation flow8.1.2 Automating a conversation flow test8.1.3 Testing the dialogue flowchart8.1.4 Testing the unexpected error paths8.2 Nonfunctionally testing a dialogue flow8.2.1 User experience testing8.2.2 Load testingSummary
Part 4. Maintenance
9 Deployment and management
9.1 Where to store your code9.1.1 Taking the Wild West approach9.1.2 Using source control for code9.2 Where to run your code9.2.1 Development environment9.2.2 Test environment9.2.3 Production environment9.2.4 After the first production deployment9.3 Using source control for other assetsSummary
10 Improving your assistant
10.1 Using a success metric to determine where to start improvements10.1.1 Improving the first flow to fix containment problems10.1.2 Inspecting other process flows for containment problems10.2 Analyzing the classifier to predict future containment problems10.2.1 Representative baseline10.2.2 Finding gaps in the training data10.3 When and why to improve your assistant10.3.1 You can’t fix everything at once10.3.2 You can’t always predict how users will react10.3.3 User needs will change10.3.4 Not every problem is technicalSummary
Part 5. Advanced/optional topics
11 Building your own classifier
11.1 Why build your own classifier?11.1.1 Classification is a differentiator11.1.2 Classification is a core competency11.1.3 Traceability11.1.4 To learn11.1.5 Build or buy?11.2 Build a simple classifier from first principles11.2.1 The simplest text classifier11.2.2 The mathematics behind a simple classifier11.3 Expanding the simple classifier11.3.1 Predicting more than one class11.3.2 An all-in-one classifier11.3.3 Comparing binary classifiers to all-in-one classifiers11.4 Extending even further11.4.1 What happens when you add more training data?11.4.2 Exercise: Experiment on your ownSummary
12 Additional training for voice assistants
12.1 Collecting data to test a speech-to-text model12.1.1 Call recordings as speech training data12.1.2 Generating synthetic speech data12.2 Testing the speech-to-text model12.2.1 Word error rate12.2.2 Intent error rate12.2.3 Sentence error rate12.3 Training a speech-to-text model12.3.1 Custom training with a language model12.3.2 Custom training with an acoustic model12.3.3 Custom training with grammarsSummary
Appendix. Glossary of terms and abbreviations
index
inside back cover

Content preview from Conversational AI

9 Deployment and management

This chapter covers

Tracking changes to your AI assistant over time
Managing multiple versions of your assistant
Safely deploying code to production

The day Fictitious Inc. decided to build a conversational AI assistant was the happiest day of John Doe’s life. John, a developer at Fictitious Inc., could not wait to get started building. He raced ahead of the rest of his project team, spending nights and weekends developing the assistant. John’s project team was happy about his enthusiasm, but nobody on the team was exactly sure what John was up to all the time.

Communication with John was a struggle. He was slow to respond to instant messages and emails. Occasionally a team member would walk to John’s desk find ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781617298837Publisher Support Publisher Website Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Conversational AI

by Andrew Freed

9 Deployment and management

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.