An initial release of a large language model (LLM) makes for a nice marketing moment, but value lies in the work you do to make something a true "1.0"-level product experience. In this report, Phillip Carter, who spearheads AI initiatives at Honeycomb, provides an introduction to using observability tools and practices that will help you improve modern LLM and AI products after they've been released.
MLOps professionals, SREs, software engineers, developers, and architects will learn not only the importance of OpenTelemetry, but also the methods of feeding observability data back into development. This report is also ideal for CTOs and other senior-level practitioners in your organization.
- Why observability is essential to taking an AI product from 0.1 to 1.0
- Strategies for implementing good observability for LLMs using standards including OpenTelemetry
- How observability isn't just a way to get better reliability in production, but how it can feed back into core development workflows
About the author:
Phillip Carter is a principal product manager at Honeycomb, leading the company's AI and OpenTelemetry initiatives. He is also a maintainer of the OpenTelemetry project, the de facto standard for observability instrumentation.
Table of contents
Observability for Large Language Models
- Why Observability Matters for LLMs
- A Quick Primer on Prompt Engineering
- A Quick Primer on OpenTelemetry
- Designing Your Telemetry
- Cost Is Important but Usually Not Critical
- It’s All About Inputs and Outputs
- Tracking Latency and Errors for API Calls to LLMs
- Monitoring and Service-Level Objectives
- Feeding Observability Data Back into Development
- Observability Is a Team Sport
- Title: Observability for Large Language Models
- Release date: September 2023
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781098159740
You might also like
Pretrain Vision and Large Language Models in Python
Master the art of training vision and large language models with conceptual fundaments and industry-expert guidance. …
Introduction to Transformers for NLP: With the Hugging Face Library and Models to Solve Problems
Get a hands-on introduction to Transformer architecture using the Hugging Face library. This book explains how …
Designing Large Language Model Applications
Transformer-based language models are powerful tools for solving a variety of language tasks and represent a …
Language Models in Plain English
Recent advances in machine learning have lowered the barriers to creating and using ML models. But …