August 2015
Intermediate to advanced
312 pages
7h 28m
Japanese
Content preview from 初めてのSpark
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
196
10 章 Spark Streaming
10
回のデータのバッチごとに動作するように設定します。失われたデータを回復する場合には、
Spark Streaming
は最後のチェックポイントまで戻れば良いのです。
この後本章では、
Spark Streaming
の変換、出力操作、入力ソースの詳細を見ていきます。その
後、フォールトトレランスやチェックポイント処理に戻って、常時稼働のためのプログラムの設定
方法について説明します。
10.3
DStream
の変換は、ーなものと、ーなものと分類されます。
•
ーでは、各バッチの処理は過去のバッチのデータに依存しません。ス
テートレスな変換には、
3
から
4
で見てきた
map()
、
filter()
、
reduceByKey()
と
いった、一般的な
RDD
の変換が含まれます。
•
対照的に、ーでは、現在のバッチの処理結果を計算するために、過去の
バッチのデータや中間結果が使われます。ステートフルな変換には、スライディングウィ
ンドウや、時間の経過と共に追跡される状態に基づく変換が含まれます。
10.3.1
テートレな
10-1
に、ステートレスな変換の一部を示します。ステートレスな変換は、各バッチ、すなわ
ち
DStream
中の各
RDD
に対して適用されるシンプルな
RDD
の変換です。すでに
10-3
では
filter()
を見ました。
3
および
4
で議論した
RDD
の変換の多くは、
DStream
でも利用でき
ます。注意が必要なのは、 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.