July 2020
Intermediate to advanced
262 pages
5h 34m
Chinese
Content preview from 大规模数据分析和建模：基于 Spark 与 R
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
调试
｜
151
表
9-1
：连接
Spark
所用的参数（续）
名称
取值
app_name
运行在 Spark 集群中时使用的应用名称
version
使用的 Spark 版本
。
这个参数只对
"local"
和
"livy"
连接有效
config
为生成的 Spark 连接定制配置属性
。
详见
spark_config
可以通过在 config 参数中指定列表来配置其他设置。现在让我们看看这些设置有什么。
9.2.2
提交设置
运行 spark-submit（启 动 Spark 的终端应用程序）时必须指定某些设置。例如，由
于 spark-submit 启动一个作为 Java 实例运行的驱动节点，需要将分配的内存指定为
spark-submit 的参数。
你可以通过运行以下命令列出所有可用的 spark-submit 参数：
spark_home_dir() %>% file.path("bin", "spark-submit") %>% system2()
为了简单易读，我们在表 9-2 中提供了这个命令的输出，用适当的 spark_config()
设置替换 spark-submit 参数，并删除了不适用或本章中已经介绍的参数。
表
9-2
：配置
spark-submit
的可用设置
名称
取值
sparklyr.shell.jars
作为
spark_connect()
中的
jars
参数
给定
sparklyr.shell.packages
包含在驱动节点和执行节点类路径上的
JAR 的 Maven 坐标逗号分隔列表
。
将搜索
本地 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.