October 2025
Intermediate
424 pages
10h 19m
Korean
Content preview from 실무로 통하는 LLM 애플리케이션 설계
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
238
2
부
LLM 활용하기
요약하자면, 지속적 사전 훈련은 도메인 특화 텍스트가 충분히 많고 해당 도메인이 고유한 언
어 구조나 어휘를 갖는 경우에 특히 효과적입니다. 또한 지속적 사전 훈련은
LLM
을 새로운 언
어에 적응시키는 데도 활용할 수 있습니다.
TIP
도메인 특화 텍스트에는 해당 분야에서만 사용하는 전문 용어
jargon
가 포함될 때가 많습니다. 필자가 효과를 본
전략 중 하나는 해당 전문 용어들을 별도의 토큰으로 추가해 모델에 반영하는 것입니다.
지속적 사전 훈련에는 상당한 연산 자원이 필요하지만, 소규모 데이터셋에 대한 파인 튜닝은
훨씬 적은 자원으로 수행할 수 있습니다. 오늘날처럼 거대 언어 모델이 보편화된 시대에는 연
산과 메모리 자원 소모를 줄이는 다양한 방법을 모색하는 것이 중요합니다.
다음으로는 제한된 자원 환경에서도 파인 튜닝을 가능하게 하는 파라미터 효율적 파인 튜닝
(
PEFT
) 기법들을 살펴보겠습니다.
7.2
파라미터 효율적 파인 튜닝(
PEFT
)
PEFT
의 핵심은 전체 모델 파라미터를 모두 업데이트하는 대신, 일부만 선택적으로 업데이트
함으로써 연산과 저장 비용을 크게 줄이는 데 있습니다.
현재 널리 사용되는
PEFT
기법들은 크게 다음 세 가지로 나눌 수 있습니다.
●
새로운 파라미터 추가: 기존 모델에 추가 파라미터를 삽입하고, 지속적 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.