April 2026
Intermediate
592 pages
13h 28m
Korean
Content preview from 에이전트 시대의 AI 시스템 설계
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
484
에이전트 시대의 AI 시스템 설계
TTL
을
5
분으로 설정하지만, 더 길게 설정하는 것이 바람직할 때도 많다. 모델의 버전이 바뀌면 생성 결과도 크게
달라지기 마련이므로 전체 캐시를 무효화해야 한다.
클라이언트 쪽 캐싱 대 서버 쪽 캐싱
오픈
AI
, 구글, 앤트로픽 같은 모델 제공업체의 암묵적인 서버 쪽 캐싱은 비용을 절약하지만, 지연시간 개선은 제
한적이다. 캐시를 사용해서 초기 상태를 로드할 뿐 응답 자체는 새로 생성하는 것이라서,
TTFT
가 빨라지긴 하지
만 전체적인 지연시간이 획기적으로 줄지는 않는다. 빈번한 요청에 대해 전체 지연시간을 줄이는 것이 목표라면
전체 응답 생성 과정을 단축하는 클라이언트 쪽 캐싱이 낫다.
8.2.5
참고문헌
GPTCache
프레임워크 (
https
://
oreil
.
ly
/
knrd8
)는 임베딩 기반 유사성을 사용해서 클
라이언트 쪽 의미 캐시를 구현한다. 길
Gill
등의
2024
년 논문 (
https
://
arxiv
.
org
/
abs
/
2403
.
02694v3
)은 연합 학습을 이용해서 사용자 개인정보를 보호하는 사용자 중심 의미 캐
싱 구축 방법을 논의한다. 자동 접두어 캐싱은 자
Jha
와 왕
Wang
의 수업 프로젝트에서 소개되었고
(
2023
,
https
://
oreil
.
ly
/
gEWYS
)
vLLM
(
https
://
oreil
.
ly
/
8pe9V
)에 구현되었다.
앤트로픽의 블로그 글 ‘
Prompt
caching
with
Claude
’(
https
://
oreil
.
ly
/
Cfx0V
)는 노
션
Notion
이 클로드의 프롬프트 캐싱을 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.