May 2024
Beginner to intermediate
549 pages
8h 11m
Chinese
book
Spark机器学习实战
by Posts & Telecom Press, Siamak Amirghodsi, Meenakshi Rajendran, Broderick Hall, Shuen Mei
Content preview from Spark机器学习实战Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第4章 构建一个稳健的机器学习系统的常用攻略
在这一章，将讨论以下内容：
- 借助Spark的基本统计API构建属于自己的算法；
- 用于真实机器学习应用的ML管道；
- 使用Spark标准化数据；
- 划分数据为训练集和测试集；
- 新Dataset API的常见操作；
- 使用Spark 2.0从文本文件创建和使用RDD、DataFrame和Dataset；
- Spark ML的LabeledPoint数据结构；
- 使用Spark 2.0访问Spark集群；
- 使用Spark 2.0之前的版本访问Spark集群；
- 在Spark 2.0中使用SparkSession对象访问SparkContext；
- Spark 2.0中的新模型导出和PMML标记；
- 使用Spark 2.0进行回归模型评估；
- 使用Spark 2.0进行二分类模型评估；
- 使用Spark 2.0进行多标签分类模型评估；
- 使用Spark 2.0进行多类分类模型评估；
- 在Spark 2.0中使用Scala Breeze库处理图像。
4.1 引言
在每一个业务领域，例如从运营小型企业到创建、管理关键任务应用程序，许多常见的任务在执行功能的过程中由于某种原因，需要被嵌入到工作流中作为其中的一部分。对于构建稳健的机器学习系统也是如此。在 Spark 机器学习中，一些任务的流程包括模型开发（训练、测试和验证）阶段的数据划分、输入特征向量数据的标准化、使用Spark API创建ML管道。我们在本章中提供了一组攻略，使读者能够接触到实现端到端机器学习系统的实际需求。
本章的目的是演示一些存在于任何稳健的 Spark 机器学习系统实现中的常见任务。为了避免在本书的每一个单独攻略中重复引用这些常见任务，我们在本章中将这些常见任务单独作为简短攻略呈现，读者可以在阅读其他章节时根据需要再进行引用。这些攻略可以单独使用，也可以包含在更大的系统中作为一个管道子任务。需要注意的是，尽管这些常见攻略可能会在后面章节中的机器学习算法中进一步重点阐述，但为了内容的完整性，仍会将它们作为独立攻略包含在本章中。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.