November 2021
Intermediate to advanced
340 pages
10h 46m
Chinese
Content preview from Spark快速大数据分析（第2版）
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
200
｜
第
8
章
表8-2：可以写入Kafka输出池的DataFrame的表结构
列 名 列类型 说 明
key
（可选）
string
或
binary
如果有，那么这些字节会写为
Kafka
记录的键，否则
键为空
value
（可选）
string
或
binary
这些字节会写为
Kafka
记录的值
topic
（只有选项中没有指定
"topic"
时是必选的）
string
如果选项中没有指定
"option"
，那么这个字段用于决
定将键
−
值对写到哪个主题。这对于将写操作打散到
多个主题很有用。如果在选项中指定了
"topic"
，那么
这个字段的值会被忽略
在写入
Kafka
时，
3
种输出模式都是受支持的，但不推荐使用完整模式，因为相同的记录
会重复输出。下面这个示例可以将我们之前的单词计数查询结果以更新模式写入
Kafka
。
# Python代码
counts = ... # DataFrame[word: string, count: long]
streamingQuery = (counts
.selectExpr(
"cast(word as string) as key",
"cast(count as string) as value")
.writeStream
.format("kafka")
.option("kafka.bootstrap.servers", "host1:port1,host2:port2")
.option("topic", "wordCounts")
.outputMode("update") ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.