Chapter 2. The Structured Context Interface for Governance and Trust
In just a few years, prompt engineering has turned into a cottage industry. There are many books on the topic, as well as seemingly endless blogs and LinkedIn posts. Then there are the workshops, YouTube videos, and courses.
The major LLM providers, such as OpenAI, Anthropic, and Google, have published their own guides. For example, Google has written a 68-page handbook by Lee Boonstra. It covers topics like how to change the creativity of responses and craft prompts for multistep reasoning.
Yet all this points to something interesting: prompt engineering is a clear sign of the limitations of chat-based LLMs. They are brittle, as they rely on probabilistic transformer models. After all, with an application like ChatGPT, Claude, or Gemini, you will likely get a different response with the same prompt. Even minor changes in a word or punctuation can result in wildly different outputs.
True, this unpredictability is fine for summarizing long PDF documents or evaluating customer reviews. But it is far from ideal when it comes to working with complex enterprise workflows, which need to be reliable and consistent.
As for agentic AI, it certainly holds much promise to address the issues of chat-based interfaces. This should mean that prompt engineering will fade in importance. But successfully deploying agentic AI systems will be challenging. According to a report from Gartner, more than 40% of these projects will be ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access