January 2019
Beginner to intermediate
576 pages
14h 31m
Chinese
Content preview from 深度學習｜內行人的做法
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
深度網路的一般架構原則
|
107
L-BFGS
演算法在實務中使用的情況
雖然 L-BFGS 法具有一些相當有趣的特性，但它在深度網路實務中並不常
使用。
共軛梯度法。 共軛梯度法是根據共軛訊息，來引導線搜索處理行進方向。共軛梯度法
著重於最小化共軛 L2 範數。共軛梯度法非常類似於梯度遞減法，因為它們都是採用線
搜索的做法。它們之間主要區別在於，共軛梯度法要求線搜索過程中的每個連續步驟，
必須相對於方向彼此形成共軛的關係。
無海森矩陣法。 無海森矩陣最佳化演算法與牛頓法有關，但它可以讓我們所得到的二
次函數最小化效果更好。它是 James Martens 於 2010 年針對神經網路所發展出來的強大
最佳化方法。我們會採用共軛梯度迭代方法，來找出二次函數的最小值。
超參數
我們在這裡把超參數（hyperparameter）定義為，使用者可自由選擇、有可能影響模型
訓練效能表現的任何配置設定值。
超參數可分為以下幾類：
• 層的大小
• 幅度（動量、學習速率）
• 正則化（隨機拋棄 [dropout]、拋棄連結 [drop connect]、L1、L2）
• 激活函數（以及同類的函數）
• 權重初始化策略
• 損失函數
• 訓練期間的階段設定（小批量的大小）
• 輸入資料（向量化）的歸一化方案
我們將在本節介紹一些新的深度學習訓練相關超參數，進一步擴展第 1 章的概念。
108
|
第三章：深度網路基礎
關於超參數的幾點特別說明
有一些超參數只適用於某些特定的情況。我們在第 6 章和第 7 章這兩個章
節中，會更進一步詳細說明。此外，變動特定超參數有可能會影響到其他 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.