November 2017
Intermediate to advanced
328 pages
6h 51m
Japanese
Content preview from 入門 PySpark ―PythonとJupyterで活用するSpark 2エコシステム
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
134
7
章
GraphFrames
7.4
グラフの構築
これでデータはインポートできたので、グラフを構築しましょう。そのためには、頂点と辺のための
構造を構築します。本書の執筆時点では、GraphFramesでは頂点と辺のために特別な命名法に従わな
ければなりません。
●
頂点を表す列は、
id
という名前を持たなければなりません。このセクションの例では、フライト
データで頂点となるのは空港です。したがって、
airports
DataFrame中の IATA 空港コード
の名前は
id
に変更しなければなりません。
●
辺を表す列には、始点（
src
）と終点（
dst
）が必要です。フライトデータの場合フライトが辺
になるので、
departureDelays_geo
DataFrameの
src
と
dst
が始点と終点の列になります。
グラフの辺をシンプルにするために、
departureDelays_Geo
DataFrame 内の列の一部を抜き
出して
tripEdges
DataFrameを作成します。同様に、GraphFramesの命名規則に合わせるために
IATA
列の名前を
id
に変更しただけの
tripVertices
DataFrameも作成します。
# Spark
の
GraphFrames
パッケージが
#
インストール済みなのを確認しておくこと
from pyspark.sql.functions import *
from graphframes import *
#
頂点（空港）と辺（フライト）の作成
tripVertices ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.