Large Language Models: The Hard Parts

Preface

An alternative title for this book could have been LLMs Behaving Badly. If you come from a background in financial modeling, you may have noticed the parallel with Emanuel Derman’s seminal work Models. Behaving. Badly.¹ Just as Derman cautioned against treating financial models as perfect representations of reality, this book aims to highlight the limitations and pitfalls of large language models (LLMs) in practical applications.

Like financial models that failed to capture the complexity of human behavior and market dynamics, LLMs have inherent constraints. They can hallucinate facts, struggle with logical reasoning, and fail to maintain consistency in long outputs. Their responses, while often convincing, are probabilistic approximations based on training data rather than true understanding, even though humans insist on treating them as “machines that can reason.”

In recent years, LLMs have emerged as a transformative force in technology. From ChatGPT and Gemini to Claude and Mistral, these systems have captured the public’s imagination and sparked a gold rush of AI-powered applications. However, beneath this technological revolution lies a complex landscape of challenges that developers, data scientists, and technical leaders must navigate.

We wrote this book because we’re optimistic about the power and possibilities of LLMs but realistic about how hard it is to deploy them successfully, widely, and reliably. This book focuses on bringing awareness to key LLM challenges ...