Quick Start Guide to Large Language Models: Strategies and Best Practices for ChatGPT, Embeddings, Fine-Tuning, and Multimodal AI, 2nd Edition

8

AI Alignment: First Principles

Introduction

The past few chapters have dealt mostly with teaching AI models to solve tasks on our behalf through fine-tuning with labeled data and some more advanced prompting techniques, such as grabbing dynamic few-shot examples with semantic search. As we wrap up the second part of this book, it’s time to step back and take a look at a modern AI paradigm that’s actually not so much of a modern idea—alignment.

Alignment doesn’t have a strict technical definition, nor is it an algorithm that we can simply implement. In broad terms, alignment refers to any process whose goal is to instill/encode behavior of an AI that is in line with the human user’s expectations. Wow, that’s broad, right? It’s supposed to ...