November 2021
Intermediate to advanced
340 pages
10h 46m
Chinese
Content preview from Spark快速大数据分析（第2版）
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
138
第 6 章
Spark SQL
与
Dataset
第
4
章和第
5
章介绍了
Spark SQL
和
DataFrame API
。我们探究了如何连接到内建数据源
和外部数据源，体验了
Spark SQL
引擎，探索了
SQL
和
DataFrame
的互操作性和高级转化
操作，还创建并管理了视图和表。
第
3
章已经简单介绍过
Dataset
API
，不过我们略过了一个重要环节，那就是如何在
Spark
中创建、存储、序列化和反序列化
Dataset
这种强类型的分布式集合。
本章将深入
Dataset
内部来帮助你理解它
。我们会探索如何用
Java
和
Scala
操作
Dataset
，
Spark
如何管
理内存来存放
Dataset
这种高级
API
的数据结构，以及如果管理使用
Dataset
所产生的相关开销。
6.1
Java
与
Scala
共用一套
API
第
3
章（ 图
3-1
和表
3-6
）介绍过，
Dataset
为强类型对象提供了统一的公用
API
。在
Spark
支持的语言中
，只有
Scala
和
Java
是强类型的，因此，
Python
和
R
只支持无类型的
DataFrame API
。
Dataset
是领域专用的有类型对象，可用函数式编程或
DSL
算子来并行操作。
有了这套共用的
API
，
Java
开发人员不会再落后一步
。比如，当
Scala
的
groupBy()
、
flatMap()
、
map()
、
filter()
等
API
发生接口或行为变化时，
Java
也会同步变化，因为这
是
Java
实现与
Scala
实现共用的统一接口。
Spark SQL
与
Dataset
｜
139
用于
Dataset
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.