October 2014
Intermediate to advanced
288 pages
6h 27m
Japanese
Content preview from 実践 機械学習システム
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
240
12
章 ビッグデータ
12.2.3
内部で行われていること
jugはどのように動いているのでしょうか？ 基本的にはとてもシンプルです。タスクは関数と引数か
ら構成されています。引数は値か他のタスクのどちらかを取ります。もし、あるタスクが他のタスクを
引数に取れば、その二つのタスク間には依存関係が作られます（一つ目のタスクの結果が準備されるま
では、二つ目のタスクは実行できません）。
上記の内容に基づき、各タスクのためのハッシュを再帰的に計算していきます。ハッシュとは、そこ
に辿り着くまでに計算した全作業をエンコードして、数値化したものです。
jug execute
を実行した
とき、次のコードに示すような簡単なループ処理が行われています。
for t in alltasks:
if t.has_not_run() and not backend_has_value(t.hash()):
value = t.e
xecute()
save_to_backend(value, key=t.hash())
実際のループ処理では、並列処理のロック問題のため、より複雑になりますが、基本的なアイデア
は上のコードに示した通りです。
キャッシュデータは、標準ではハードディスク（
jugfile.jugdata/
という名前のディレクトリ）
にファイルとして書き込まれます。また、他の選択肢としてRedisデータベースを用いることも可能で
す。
適切なロッキング処理をjug が行うことで、複数のプロセッサ上でタスクを実行する ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.