April 2026
Intermediate
592 pages
13h 28m
Korean
Content preview from 에이전트 시대의 AI 시스템 설계
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
494
에이전트 시대의 AI 시스템 설계
8.4.1
문제
LLM
기반
AI
애플리케이션에서 성능 테스트는 보통 주어진 인퍼런스 구성
inference
setup
4
의 성능
을 테스트하는 것으로 귀결된다.
AI
애플리케이션이 전통적인 서버 애플리케이션과 비슷해 보
이겠지만, 전통적인 부하 테스트
load
test
를 수행하는 것만으로는 충분하지 않다. 전통적인
부하
테스트
는
HTTP
400
/
500
오류 같은 서버 실패 지점
failure
point
을 찾는 데 주력하지만,
AI
애플
리케이션에서는 성능을 더 세밀하게 파악할 필요가 있다. 특히, 단순히
어디에서
실패가 발생
하는지를 밝히는 것뿐만 아니라 서비스 품질이
어떻게
떨어지기 시작하는지를 정확히 파악할
수 있어야 한다.
다양한 성능 저하 지점과 제약 조건을 이해하는 것은 전체 애플리케이션 개발 및 운영에 중요
하다. 예를 들어
LLM
인퍼런스의 핵심 목표 중 하나는 응답을 빠르게 생성하는 것이다. 응답이
0
.
3
초 이내에 생성되어야 한다고 정해 놓고 요청의
95
%가 이 임곗값보다 빠르게 처리되면 테
스트를 통과한 것으로 간주하는 것으로는 충분하지 않다. 대신, 요청의
5
% 이상이 느려지는 저
하 지점과 애플리케이션이 이 저하 지점으로 다가가게 만드는 제약 조건들을 식별해야 한다.
응답 생성 속도의 경우 요청 수가
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.