Frank Kane's Taming Big Data with Apache Spark and Python

Special MLlib data types

One thing you need to understand with MLlib is that it does have a few special data types. For example, it has a Vector type that can be either dense or sparse. Sparse vectors can be useful if you have a matrix in machine learning, where most of the cells are empty. It's a more compact way of representing vectors or arrays of data where a lot of the values have nothing in them. This actually happens pretty often when you're doing things such as recommendations. It also has LabeledPoint, allowing you to attach some sort of meaning to a data point. Finally, in the context of recommendations, it has a Rating type that we'll look at later in this section.