August 2022
Beginner to intermediate
424 pages
7h 18m
Japanese
book
機械学習エンジニアのためのTransformers ―最先端の自然言語処理ライブラリによるモデル開発
by Lewis Tunstall, Leandro von Werra, Thomas Wolf, 中山 光樹
Content preview from 機械学習エンジニアのためのTransformers ―最先端の自然言語処理ライブラリによるモデル開発
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
まえがき
2017年の登場以来、Transformerは、学術界と産業界の両方で幅広い自然言語処理（NLP）タスクに取り組むためのデファクトスタンダードとなっています。そのため、気づかぬうちに使っていることもあります。たとえば、Googleはユーザーの検索クエリをより深く理解し、検索エンジンを改善するためにBERTを使っています。同様に、OpenAIのGPT系のモデルは、人間が生み出したようなテキストや画像を生成する能力を持っているとメディアで何度も取り上げられてきました†1。GitHub Copilot（https://copilot.github.com）のようなアプリケーションでも使われています。Copilotでは、図1に示すように、コメントをソースコードに変換できます。
[†1] NLPの研究者は、自分の仕事に「セサミストリート」の登場人物の名前を付ける傾向があります。これらの略語の意味については、「1章 入門 Transformers」で説明します。
図1: GitHub Copilotの例。タスクの簡単な説明を与えると、クラス全体を提案する（class以下はすべて自動生成）
では、この分野を一夜にして変えてしまったTransformerとは、いったい何なのでしょうか。多くの偉大な科学的ブレークスルーがそうであるように、それはアテンション、転移学習、ニューラルネットワークのスケールアップといった、当時の研究コミュニティで浸透していたアイデアを組み合わせたものでした。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.