November 2021
Intermediate to advanced
340 pages
10h 46m
Chinese
Content preview from Spark快速大数据分析（第2版）
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
下载并开始使用
Apache Spark
｜
29
Databricks
社区版
Databricks
公司在云上提供受管理的
Spark
平台服务。除了在本地机器上运行本地模式
的
Spark
，你也可以使用免费的
Databricks
社区版（见图
2-10
）来尝试本章和其他章节
中的一些示例。作为
Spark
的学习工具，社区版包含很多值得一看的指引和示例。除
了能用
Python
、
R
、
Scala
或
SQL
编写自己的代码笔记本，你也可以导入包括
Jupyter
笔记本在内的其他代码笔记本。
图 2-10：Databricks 社区版
请访问
Databricks
网站，按照说明即可获取账号来免费试用。
2.6
第一个独立应用
为了帮助你学习和探索，
Spark
发行版的每个组件都自带了一组示例程序。你不妨在安装
路径下找到
examples
文件夹，看看其中到底有些什么。
你可以从本地机器上的安装目录运行
Spark
自带的任意
Java
或
Scala
版本的示例程序
，使
用命令
bin/run-example
<class> [params]
即可，举例如下：
$
./bin/run-example JavaWordCount README.md
执行以上这条命令时，控制台会打印出很多
INFO
级别的消息、
README.md
文件中的所有
单词，以及它们的出现次数（单词计数是最简单的分布式计算任务）。
2.6.1
统计
M&M
巧克力豆
前面的示例统计了文件中的单词数。如果这个文件特别大，那么它会跨节点分布在集群中，
分区为多个小数据块。然后，我们的
Spark
程序会将任务分发到集群 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.