April 2020
Intermediate to advanced
816 pages
18h 32m
Chinese
Content preview from 精通機器學習
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
用多個裝置訓練模型
|
675
•
你可以建立一個高效的整體
：
把不同的訓練好的模型放在各個
GPU，
就可以更快速地
取得所有的預測
，
來產生整體的最終預測
。
但如果你想要用多個
GPU
訓練
一個模型呢
？
用多個裝置訓練模型
用多個裝置訓練一個模型的主要方法有兩種
：
模型平行化
，
也就是將模型拆開並放到各
個裝置
，
以及
資料平行化
，
也就是在各個裝置複製模型
，
並且用資料的子集合來訓練各
個複本
。
我們先仔細瞭解這兩個選項
，
再用多個
GPU
訓練模型
。
模型平行化
到目前為止
，
我們都用一台裝置來訓練各個神經網路
。
如果我們想要用多台裝置訓練一個
神經網路呢
？
此時你必須將模型拆成不同的區塊
，
並且在不同的裝置上執行各個區塊
。
遺憾的是
，
這種模型平行化非常麻煩
，
而且跟神經網路的結構有很大的關係
。
對全連接網
路而言
，
這種做法通常不會帶來太多好處
（
見圖
19-15）。
直觀地說
，
分割模型最簡單的
做法是將每一層放在不同的裝置
，
但是這種做法是無效的
，
因為每一層都必須等待上一層
的輸出才能做事
，
或許你可以將它直向切開
，
例如
，
把每一層的左半邊放到一個裝置
，
右
半邊放到另一個裝置
？
這種做法比較好
，
因為各層的兩邊確實可以平行運作
，
但問題是
，
下一層的兩邊都需要這一層的兩邊的輸出
，
所以會有許多跨裝置通訊
（
以虛線表示
）。
這
可能會完全抵消平行運算的好處
，
因為跨裝置通訊很慢
（
尤其是當裝置位於不同的電腦
時
）。
直向切開
不太好⋯
全連接神經網路
每個裝置一層
很糟！
圖
19-15
拆開全連接神經網路
676
|
第十九章：大規模訓練與部署 TensorFlow 模型
有些神經網路結構 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.