January 2026
Intermediate
404 pages
7h 22m
Korean
Content preview from AI 에이전트 엔지니어링
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
291
10
장
운영 환경 모니터링
계층 지표 목적 예시 조치
사용자
피드백
재질의/재표현 비율 첫 시도에 이해되었는지 측정 의도 인식 개선
작업 포기율 사용자를 혼란스럽게 하거나
좌절시키는 워크플로 식별
플로 간소화 또는 명확화
프롬프트 추가
명시적 평점
(좋아요/싫어요)
시스템 유용성에 대한 정성
평가 수집
평가용 출력 선별에 활용
각 지표는 오픈텔레메트리로 로깅하고 프로메테우스나 로키에서 집계하며 그라파나로 시각화
하고 필요하면 템포 트레이스와 연동할 수 있습니다. 목표는 모든 것을 수집하는 것이 아니라,
의미 있는 변화를 탐지하는 데 필요한 것만 수집하는 것입니다. 또한 빠른 진단과 지속적 개선
을 지원하는 방식으로 운영해야 합니다.
10.2
모니터링 스택
어떤 모니터링을 선택하느냐에 에이전트 시스템의 개발 속도가 달렸습니다. 가시성은 지연시
간, 가용성 같은 기존의 인프라 지표뿐 아니라 할루시네이션률, 도구 효과, 사용자 입력의 분
포 이동 같은 의미적 신호도 포착해야 합니다. 현재 환경에서는 랭그래프, 크루
AI
, 오토젠 같
은 프레임워크와 매끄럽게 통합되는 오픈 소스 도구가 강조됩니다. 이러한 도구는 파운데이션
모델의 확률적 특성을 다루면서 분산 트레이스, 로그 수집 및 집계, 알림을 지원합니다. 많은
기업이 이미 스플렁크
Splunk
, 데이터독
Datadog ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.