AI-Native Software Engineering

Chapter 2. From LLMs to Agents

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

To use AI well in your work, you need a working mental model of what you are actually using. LLMs are statistical engines that predict the next token, and agents are those engines wrapped in a loop that lets them take action. Those two ideas, internalized properly, will take you further than any list of tips about prompt phrasing.

This chapter gives you that model. The first half explains how LLMs actually work and where they succeed or fail. The second half shows how a language model becomes an agent by adding tools, context, and a loop, and discusses how to reason about agent behavior in practice. The closing section surveys the AI tooling landscape, including the categories of tools software teams are likely to adopt, how they fit into the development lifecycle, and how to choose among them without chasing hype.

This chapter is not an introduction to machine learning. You will not find a derivation of the Transformer or a primer on backpropagation. Everything ...