January 2019
Beginner to intermediate
576 pages
14h 31m
Chinese
Content preview from 深度學習｜內行人的做法
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
396
|
第九章：在 Spark 上使用深度學習與 DL4J
透過 YARN 運行 Spark
YARN 是一種 Hadoop API，提供給一般用途的應用程式，好讓程式可以像 MapReduce
一樣，做為第一等公民來運行。目前 YARN 實作的 Spark 可支援兩種執行 Spark job 的
模式：
• yarn- 客戶端（client）
• yarn- 集群（cluster）
YARN 應用程序包含了
ApplicationMaster
與
NodeManager
這兩個概念。
YARN、ApplicationMaster 與 NodeManager
ApplicationMaster 會追蹤Hadoop 集群中特定job 的資源與執行情況。
NodeManager 則會配置 YARN 容器（container）， 讓 job 裡的 task 於容器中運
行。本節的目的並不是要介紹更多分散式系統的術語，讓你感到更加混淆，而
是想要提供一些背景知識，讓你更了解那些用於正式生產環境中的 Hadoop 集
群，其內部究竟是什麼狀況。
以 Spark 來說，我們可以看到 Spark 執行程序（executor）在 YARN 容器內部
運行，Spark 的 job 則是由 Spark ApplicationMaster 來進行調度。我們只是希望
你能瞭解一下這些流程，以便能夠更瞭解系統何時在何處如何運行 Spark job。
每個 Spark 執行程序都是以 YARN 容器的形式來運行的
4
。Spark 會在同一個容器中託管 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.