February 2025
Intermediate
376 pages
9h 25m
Japanese
Content preview from 仕組みからわかる大規模言語モデル 生成AI時代のソフトウェア開発入門
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
1
1.3.3
埋め込み層
埋め込み層では、トークンID を要素とする
n
次元ベクトルを入力とし、
n#D
model
の行列
を出力します。出力行列の各行は、入力トークンID に対応する埋め込みベクトルです。
埋め込み層の処理を詳しく見てみましょう。図 1.3.5 に埋め込み層を拡大した図を示しま
す。埋め込みの処理は、テキスト埋め込みと位置埋め込みから構成されます。はじめにテキス
ト埋め込みで、トークンID をベクトル表現に変換します。次に位置埋め込みで、各トークン
の位置情報をベクトル表現に付与します。
N
N
図1.3.5
テキスト埋め込み
テキスト埋め込みでは、トークンID を要素とする
n
次元のベクトル
x
=
[x
1
, x
2
, g, x
n
]
（各
x
i
はトークンID）を入力とし、各要素の埋め込みベクトルを出力します。このベクトル表現は
LLMのパラメータとして学習されたもので、トークン ID ごとに一つのベクトル表現が対応し
ます。出力は、
n#D
model
の行列で、各行は入力トークンID に対応する埋め込みベクトルで
す。
図1.3.6 にテキスト埋め込みの構成を示します。テキスト埋め込みの中身は
|V |#D
model
の
行列
W
E
で、各行はトークンID に対応する埋め込みベクトルです。
LLMの実装では、この行列をテーブルとみなすことができます。トークンID をインデック
スとしてこのテーブルをルックアップすることで、トークン ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.