February 2025
Intermediate
376 pages
9h 25m
Japanese
Content preview from 仕組みからわかる大規模言語モデル 生成AI時代のソフトウェア開発入門
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
のうちパラメータを持つのは、隠れ層と出力層の二つの層です。このため、このニューラル
ネットワークは2 層のニューラルネットワークと呼ばれます。
n
D
model
D
ff
D
model
図1.3.15
隠れ層
第1 層の計算は、次の数式で表すことができます。
H
=max
(
0
, XW
1
+
B
1
)
ここで、
X
は入力トークン埋め込み行列（
n#D
model
の行列）、
W
1
は重み行列（
D
model
#D
ff
の
行列）、
B
1
はバイアス項（
n#D
ff
の行列）、max(0,
$
)は ReLU 活性化関数を表しています。
ReLU活性化関数は、ニューラルネットワークに非線形性を導入する役割を果たしていま
す。線形変換だけでは、入力と出力の関係が単純な線形結合に限られてしまいます。しかし、
ReLUのような非線形関数を導入することで、ネットワークはより複雑な関数を表現できるよ
うになります。具体的には、ReLUは入力が0 以上の場合はそのまま出力し、0未満の場合は0
を出力します。この単純な処理によって、ネットワークは必要な情報を選択的に通過させ、不
要な情報を遮断することができます。これにより、ネットワークはデータ内の複雑なパターン
を学習し、より精度の高い予測を行うことが可能になります。
058
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.