March 2018
Beginner to intermediate
332 pages
6h 1m
Korean
Content preview from 9가지 사례로 익히는 고급 스파크 분석(2판): 현실 세계 빅데이터로 배우는 데이터 과학과 머신러닝
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
236
9가지 사례로 익히는 고급 스파크 분석 (2판)
val id = jso.fields.get("id")
val properties = jso.fields("properties").asJsObject.fields
val geometry = jso.fields("geometry").convertTo[RichGeometry]
Feature(id, properties, geometry)
}
}
FeatureJsonFormat
객체에
implicit
키워드를 사용한 이유는
JsValue
인스턴스 하나에 대
해
convertTo
[
Feature
] 함수를 호출했을 때
Spray
라이브러리가
FeatureJsonFormat
객체
를 찾아볼 수 있도록 하기 위함이다.
RootJsonFormat
구현의 나머지 부분은 깃허브 저장소
의
GeoJSON
라이브러리 소스 코드에서 확인할 수 있다.
8.4
뉴욕 택시 운행 데이터 준비하기
이제 스파크에서
GeoJSON
라이브러리를 이용하여 뉴욕 택시 운행 데이터를 대화식으로 분
석해볼 차례다.
HDFS
에
taxidata
디렉터리를 만들고 택시 운행 데이터를 클러스터로 복사
하자.
$ hadoop fs -mkdir taxidata
$ hadoop fs -put trip_data_1.csv taxidata/
이제 스파크 셸을 시작하자. 이때 --
jars
인수를 주어 우리에게 필요한 라이브러리를 사용할
수 있도록 한다.
$ mvn package
$ spark-shell --jars ch08-geotime-2.0.0-jar-with-dependencies.jar ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.