April 2025
Intermediate
240 pages
6h 56m
Japanese
Content preview from AI開発力を鍛える！機械学習と最適化による問題解決講座
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
154
0
1
2
3
3
深層学習における超多パラメータ最適化
3.1.3項の「最適化のパラメータも次元の呪い」において、最適化のパラメー
タ数が増えると探索空間が広大になり、最適解を得ることが困難になると説
明しました。一方で、今日の深層学習の学習パラメータの数は、数百万～数
兆と非常に膨大です。このような膨大なパラメータが作る超高次元空間で、
深層学習モデルの予測精度が最も高くなる最適解（パラメータ値の組合せ）
を探すのは、絶望的に難しく思われます。しかし、皆さんが目にするAI の予
測精度は非常に高く、学習パラメータの最適化が問題なく行われていること
がわかります。この深層学習における超多パラメータ最適化がうまく行く理
由は、大きく2つあり、なぜ深層学習が高い性能を発揮できるかともつな
がっていると考えられています。
一つ目の理由は、ニューラルネットワークの学習において目的関数が凸関数
で十分なニューロン数が存在する（自由度が高い）場合には、ニューロン数
が限られている場合に局所的最適解（ローカルミニマム）に見えたものは高
次元では接続しており、大域的最適解にもつながっていることです
[ 1 ]
。した
がって、自由度を高くすることでローカルミニマムに解がトラップされず
（ただし、平坦な場所は多く存在する）、大域的最適解にたどり着くことがで
きると言われています。
二つ目の理由は、ニューラルネットワークの学習パラメータが作る超多パラ
メータの最適化空間では、学習パラメータの初期値のすぐ近くに最適解が存
在することです
[
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.