The Developer's Playbook for Large Language Model Security

Chapter 1. Chatbots Breaking Bad

Large language models and generative AI jumped to the forefront of public consciousness with the release of ChatGPT on November 30, 2022. Within five days, it went viral on social media and attracted its first million users. By January, ChatGPT surpassed one hundred million users, making it the fastest-growing internet service in history.

However, a steady stream of security concerns emerged in the following months. These included privacy and security issues that caused companies like Samsung and countries like Italy to ban its usage. In this book, we’ll explore what underlies these concerns and how you can mitigate these issues. However, to best understand what’s going on here and why these problems are so challenging to solve, in this chapter, we will briefly rewind further in time. In doing so, we’ll see these types of issues aren’t new and understand why they will be so hard to fix permanently.

Let’s Talk About Tay

In March 2016, Microsoft announced a new project called Tay. Microsoft intended Tay to be “a chatbot created for 18- to 24-year-olds in the U.S. for entertainment purposes.” It was a cute name for a fluffy, early experiment in AI. Tay was designed to mimic a 19-year-old American girl’s language patterns and learn from interacting with human users of Twitter, Snapchat, and other social apps. It was built to conduct real-world research on conversational understanding.

While the original announcement of this project seems impossible ...