April 2026
Intermediate
592 pages
13h 28m
Korean
Content preview from 에이전트 시대의 AI 시스템 설계
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
367
6
장
신뢰성 개선
평가 측면에서는 실제 결과를 측정하는 것이 가장 좋은 접근법이지만, 결과에 기여하는 다른
요인들과 소위
교란 변수
confounding
variable
들을 모두 분리해 내기는 어려울 수 있다. 이런 상황에
서는 비즈니스 목표와 일치하는 이해하기 쉬운 핵심 성과 지표 (
KPI
)들을 기반으로 판단하는
것이 차선책인 경우가 많다. 다만 지표를 조작하려 드는 적대적 행위자
adversarial
actor
를 조심해야
한다. 따라서 심판형
LLM
을 사용할 때는 채점 기준이 결과나
KPI
의 대리 지표가 되게 만드는
데 공을 들여야 한다.
6.1.5
참고문헌
2022
년과
2023
년에 연구자들과 실무자 커뮤니티는 프런티어
LLM
이 임의의 채점 기준에 따
라 콘텐츠를 평가하는 데 사용할 수 있을 만큼 지시사항을 잘 따르게 되었다는 사실을 자연
스럽게 깨닫게 되었다. 우리 저자들이 찾은 최초의 학문적 언급은 바이
Bai
등의
2023
년 논문
(
https
://
arxiv
.
org
/
abs
/
2306
.
04181
)인데, 그들은 이 패턴을 조사관형 언어 모델
Langua
ge
-
Model
-
as
-
an
-
Examiner
이
라고 불렀다. 샹카르
Shankar
등은 정렬
alignment
과 일관성의 필요성을 논
의했고 (
2024
,
https
://
arxiv
.
org
/
abs
/
2404
.
12272
), 발로그
Balog
, 메츨러
Metzler
, 친
Qin
은 심판형
LLM
과 관련한 여러 편향과 주의 사항을 논의했다 (
2025
,
https
://
arxiv
.
org
/
abs
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.