August 2021
Beginner to intermediate
720 pages
15h 32m
Korean
Content preview from fastai와 파이토치가 만나 꽃피운 딥러닝
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
606
3
딥러닝의 기반 지식
다양한 옵티마이저의 구현체뿐만 아니라 다양한 예제와 테스트 코드도 있으니 확인해보기 바
랍니다.
옵티마이저를
SGD
에서
Adam
으로 바꾸면 가중치 감쇠 적용 방식이 바뀌며, 이는 중대한 영
향을 미칠 수 있습니다.
16.6
순수
SGD
라는 가정하에,
8
장에서 다룬 가중치 감쇠는 다음 파라미터 갱신법을 따릅니다.
new_weight = weight - lr*weight.grad - lr*wd*weight
공식의 마지막 부분이 가중치 감쇠라는 이름을 설명해주죠. 즉, 각 가중치는
lr
*
wd
의 배율만
큼 감쇠됩니다.
L2
는 가중치 감쇠의 다른 이름으로, 모든 가중치의 제곱의 합을 손실에 더합니다.
8
장
에서 봤듯이, 그레이디언트에 대해 다음과 같이 직접적으로 표현할 수 있습니다.
weight.grad += wd*weight
SGD
에서는 앞의 두 공식을 그대로 사용할 수 있습니다. 그러나 모멘텀,
RMSProp
,
Adam
에
서는 그렇지 않습니다. 가중치를 갱신할 때 그레이디언트에 몇 가지 추가 공식이 따라오기 때
문이죠.
대부분의 라이브러리는 두 번째 공식을 사용하지만, 일야 로실로프
Ilya
Loshchilov
와 프랭크 허
터
Frank
Hutter
의 「
Decoupled
Weight
Decay
Regularization
」
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.