O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Apache Mahout Clustering Designs

Book Description

Explore clustering algorithms used with Apache Mahout

About This Book

  • Use Mahout for clustering datasets and gain useful insights

  • Explore the different clustering algorithms used in day-to-day work

  • A practical guide to create and evaluate your own clustering models using real world data sets

  • In Detail

    As more and more organizations are discovering the use of big data analytics, interest in platforms that provide storage, computation, and analytic capabilities has increased. Apache Mahout caters to this need and paves the way for the implementation of complex algorithms in the field of machine learning to better analyse your data and get useful insights into it.

    Starting with the introduction of clustering algorithms, this book provides an insight into Apache Mahout and different algorithms it uses for clustering data. It provides a general introduction of the algorithms, such as K-Means, Fuzzy K-Means, StreamingKMeans, and how to use Mahout to cluster your data using a particular algorithm. You will study the different types of clustering and learn how to use Apache Mahout with real world data sets to implement and evaluate your clusters.

    This book will discuss about cluster improvement and visualization using Mahout APIs and also explore model-based clustering and topic modelling using Dirichlet process. Finally, you will learn how to build and deploy a model for production use.

    What You Will Learn

  • Explore clustering algorithms and cluster evaluation techniques

  • Learn different types of clustering and distance measuring techniques

  • Perform clustering on your data using K-Means clustering

  • Discover how canopy clustering is used as pre-process step for K-Means

  • Use the Fuzzy K-Means algorithm in Apache Mahout

  • Implement Streaming K-Means clustering in Mahout

  • Learn Spectral K-Means clustering implementation of Mahout

  • Who This Book Is For

    This book is for developers who want to try out clustering on large datasets using Mahout. It will also be useful for those users who don't have background in Mahout, but have knowledge of basic programming and are familiar with basics of machine learning and clustering. It will be helpful if you know about clustering techniques with some other tool.

    Style and approach

    This book is a hand's-on guide with examples using real-world datasets. Each chapter begins by explaining the algorithm in detail and follows up with showing how to use mahout for that algorithm using example data-sets.

    Table of Contents

    1. Apache Mahout Clustering Designs
      1. Table of Contents
      2. Apache Mahout Clustering Designs
      3. Credits
      4. About the Author
      5. About the Reviewers
      6. www.PacktPub.com
        1. Support files, eBooks, discount offers, and more
          1. Why subscribe?
          2. Free access for Packt account holders
      7. Preface
        1. What this book covers
        2. What you need for this book
        3. Who this book is for
        4. Conventions
        5. Reader feedback
        6. Customer support
          1. Downloading the example code
          2. Errata
          3. Piracy
          4. Questions
      8. 1. Understanding Clustering
        1. The clustering concept
          1. Application of clustering
        2. Understanding distance measures
        3. Understanding different clustering techniques
          1. Hierarchical methods
          2. The partitioning method
          3. The density-based method
          4. Probabilistic clustering
        4. Algorithm support in Mahout
        5. Clustering algorithms in Mahout
        6. Installing Mahout
          1. Building Mahout code using Maven
          2. Setting up the development environment using Eclipse
          3. Setting up Mahout for Windows users
        7. Preparing data for use with clustering techniques
        8. Summary
      9. 2. Understanding K-means Clustering
        1. Learning K-means
          1. Running K-means on Mahout
            1. Dataset selection
              1. Executing K-means
            2. The clusterdump result
        2. Visualizing clusters
        3. Summary
      10. 3. Understanding Canopy Clustering
        1. Running Canopy clustering on Mahout
          1. The Canopy generation phase
          2. The Canopy clustering phase
          3. Running Canopy clustering
          4. Using the Canopy output for K-means
        2. Visualizing clusters
        3. Working with CSV files
        4. Summary
      11. 4. Understanding the Fuzzy K-means Algorithm Using Mahout
        1. Learning Fuzzy K-means clustering
          1. Running Fuzzy K-means on Mahout
            1. Dataset
            2. Creating a vector for the dataset
            3. Vector reader
        2. Visualizing clusters
        3. Summary
      12. 5. Understanding Model-based Clustering
        1. Learning model-based clustering
          1. Understanding Dirichlet clustering
          2. Topic modeling
        2. Running LDA using Mahout
          1. Dataset selection
          2. Steps to execute CVB (LDA)
        3. Summary
      13. 6. Understanding Streaming K-means
        1. Learning Streaming K-means
          1. The Streaming step
          2. The BallKMeans step
        2. Using Mahout for streaming K-means
          1. Dataset selection
          2. Converting CSV to a vector file
          3. Running Streaming K-means
        3. Summary
      14. 7. Spectral Clustering
        1. Understanding spectral clustering
          1. Affinity (similarity) graph
          2. Getting graph Laplacian from the affinity matrix
            1. Eigenvectors and eigenvalues
          3. The spectral clustering algorithm
            1. Normalized spectral clustering
        2. Mahout implementation of spectral clustering
        3. Summary
      15. 8. Improving Cluster Quality
        1. Evaluating clusters
          1. Extrinsic methods
          2. Intrinsic methods
        2. Using DistanceMeasure interface
        3. Summary
      16. 9. Creating a Cluster Model for Production
        1. Preparing the dataset
        2. Launching the Mahout job on the cluster
        3. Performance tuning for the job
        4. Summary
      17. Index