November 2017
Intermediate to advanced
328 pages
6h 51m
Japanese
Content preview from 入門 PySpark ―PythonとJupyterで活用するSpark 2エコシステム
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
9.3
データの抽象化
183
9.3.3
ファイルの利用
DataShapeオブジェクトは、直接
.csv
ファイルから生成することもできます。この例では、メリー
ランド州のモントゴメリー群で起きた 404,536 件の交通違反を含むデータセットを使います。
データは
https://catalog.data.gov/dataset/traffic-violations-56dda
から
2016
年
8
月
23
日に
ダウンロードしました。このデータセットは日々更新されているので、読者の皆さんが最新の
データセットを入手したなら交通違反の数は異なっているかもしれません。
データセットはローカルの
../Data
フォルダに保存します。ただし、このデータセットは MongoDB
に保存できるよう、多少手を入れてあります。元々の形のまま日付の列付きで MongoDBからデータを
読もうとするとエラーになります。この問題については Blazeのバグを登録しました（https://github.
com/blaze/blaze/issues/1580）。
import odo
traffic = bl.Data('../Data/TrafficViolations.csv')
データセット中の列の名前がわからない場合は、DataShape から取得できます。次のコマンドですべ
てのフィールドのリストが得られます。
print(traffic.fields)
pandas
に慣れている人であれば、
.fields
と
.columns
属性が
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.