Mastering Machine Learning with Spark 2.x

Frequent pattern mining

When presented with a new data set, a natural sequence of questions is:

• What kind of data do we look at; that is, what structure does it have?
• Which observations in the data can be found frequently; that is, which patterns or rules can we identify within the data?
• How do we assess what is frequent; that is, what are the good measures of relevance and how do we test for it?

On a very high level, frequent pattern mining addresses precisely these questions. While it's very easy to dive head first into more advanced machine learning techniques, these pattern mining algorithms can be quite informative and help build an intuition about the data.

To introduce some of the key notions of frequent pattern mining, let's first ...