Semi-Supervised and Unsupervised Machine Learning: Novel Strategies

Semi-Supervised and Unsupervised Machine Learning: Novel Strategies

by Amparo Albalate, Wolfgang Minker
Released January 2011
Publisher(s): Wiley
ISBN: 9781848212039

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

This book provides a detailed and up-to-date overview on classification and data mining methods. The first part is focused on supervised classification algorithms and their applications, including recent research on the combination of classifiers. The second part deals with unsupervised data mining and knowledge discovery, with special attention to text mining. Discovering the underlying structure on a data set has been a key research topic associated to unsupervised techniques with multiple applications and challenges, from web-content mining to the inference of cancer subtypes in genomic microarray data. Among those, the book focuses on a new application for dialog systems which can be thereby made adaptable and portable to different domains. Clustering evaluation metrics and new approaches, such as the ensembles of clustering algorithms, are also described.

Table of contents

  1. Cover
  2. Title Page
  3. Copyright
  4. Part 1: State of the Art
    1. Chapter 1: Introduction
      1. 1.1. Organization of the book
      2. 1.2. Utterance corpus
      3. 1.3. Datasets from the UCI repository
        1. 1.3.1. Wine dataset (wine)
        2. 1.3.2. Wisconsin breast cancer dataset (breast)
        3. 1.3.3. Handwritten digits dataset (Pendig)
        4. 1.3.4. Pima Indians diabetes (diabetes)
        5. 1.3.5. Iris dataset (Iris)
      4. 1.4. Microarray dataset
      5. 1.5. Simulated datasets
        1. 1.5.1. Mixtures of Gaussians
        2. 1.5.2. Spatial datasets with non-homogeneous inter-cluster distance
    2. Chapter 2: State of the Art in Clustering and Semi-Supervised Techniques
      1. 2.1. Introduction
      2. 2.2. Unsupervised machine learning (clustering)
      3. 2.3. A brief history of cluster analysis
      4. 2.4. Cluster algorithms
        1. 2.4.1. Hierarchical algorithms
          1. 2.4.1.1. Agglomerative clustering
            1. 2.4.1.1.1. Comparison of agglomerative criteria
          2. 2.4.1.2. Divisive algorithms
        2. 2.4.2. Model-based clustering
          1. 2.4.2.1. The expectation maximization (EM) algorithm
            1. 2.4.2.1.1. Example: mixtures of Gaussians
        3. 2.4.3. Partitional competitive models
          1. 2.4.3.1. K-means
            1. 2.4.3.1.1. Advantages and drawbacks
          2. 2.4.3.2. Neural gas
            1. 2.4.3.2.1. Advantages and drawbacks
          3. 2.4.3.3. Partitioning around Medoids (PAM)
            1. 2.4.3.3.1. Build step
            2. 2.4.3.3.2. Swap phase
            3. 2.4.3.3.3. Advantages and drawbacks
          4. 2.4.3.4. Self-organizing maps
            1. 2.4.3.4.1. Advantages and drawbacks
        4. 2.4.4. Density-based clustering
          1. 2.4.4.1. Direct density reachability
          2. 2.4.4.2. Density reachability
          3. 2.4.4.3. Density connection
          4. 2.4.4.4. Border points
          5. 2.4.4.5. Noise points
          6. 2.4.4.6. DBSCAN algorithm
            1. 2.4.4.6.1. Advantages and drawbacks
        5. 2.4.5. Graph-based clustering
          1. 2.4.5.1. Pole-based overlapping clustering
            1. 2.4.5.1.1. Definition of a dissimilarity graph
            2. 2.4.5.1.2. Pole construction
            3. 2.4.5.1.3. Pole restriction
        6. 2.4.6. Affectation stage
          1. 2.4.6.1. Advantages and drawbacks
      5. 2.5. Applications of cluster analysis
        1. 2.5.1. Image segmentation
        2. 2.5.2. Molecular biology
          1. 2.5.2.1. Biological considerations
        3. 2.5.3. Information retrieval and document clustering
          1. 2.5.3.1. Document pre-processing
            1. 2.5.3.1.1. Word selection
            2. 2.5.3.1.2. Stop word filtering
            3. 2.5.3.1.3. Word lemmatizing/stemming
          2. 2.5.3.2. Boolean model representation
          3. 2.5.3.3. Vector space model
          4. 2.5.3.4. Term weighting
            1. 2.5.3.4.1. Term frequency component
            2. 2.5.3.4.2. Collection frequency component
            3. 2.5.3.4.3. Length normalization component
          5. 2.5.3.5. Probabilistic models
            1. 2.5.3.5.1. Binary independence retrieval model
            2. 2.5.3.5.2. The 2-Poisson model
            3. 2.5.3.5.3. Okapi weighting
        4. 2.5.4. Clustering documents in information retrieval
          1. 2.5.4.1. Clustering of presented results
          2. 2.5.4.2. Post-retrieval document browsing (Scatter-Gather)
      6. 2.6. Evaluation methods
      7. 2.7. Internal cluster evaluation
        1. 2.7.1. Entropy
        2. 2.7.2. Purity
        3. 2.7.3. Normalized mutual information
      8. 2.8. External cluster validation
        1. 2.8.1. Hartigan
        2. 2.8.2. Davies Bouldin index
        3. 2.8.3. Krzanowski and Lai index
        4. 2.8.4. Silhouette
        5. 2.8.5. Gap statistic
      9. 2.9. Semi-supervised learning
        1. 2.9.1. Self training
        2. 2.9.2. Co-training
        3. 2.9.3. Generative models
      10. 2.10. Summary
  5. Part 2: Approaches to Semi-Supervised Classification
    1. Chapter 3: Semi-Supervised Classification Using Prior Word Clustering
      1. 3.1. Introduction
      2. 3.2. Dataset
      3. 3.3. Utterance classification scheme
        1. 3.3.1. Pre-processing
          1. 3.3.1.1. Utterance vector representation
        2. 3.3.2. Utterance classification
      4. 3.4. Semi-supervised approach based on term clustering
        1. 3.4.1. Term clustering
        2. 3.4.2. Semantic term dissimilarity
          1. 3.4.2.1. Term vector of lexical co-occurrences
          2. 3.4.2.2. Metric of dissimilarity
        3. 3.4.3. Term vector truncation
        4. 3.4.4. Term clustering
        5. 3.4.5. Feature extraction and utterance feature vector
        6. 3.4.6. Evaluation
      5. 3.5. Disambiguation
        1. 3.5.1. Evaluation
      6. 3.6. Summary
    2. Chapter 4: Semi-Supervised Classification Using Pattern Clustering
      1. 4.1. Introduction
      2. 4.2. New semi-supervised algorithm using the cluster and label strategy
        1. 4.2.1. Block diagram
          1. 4.2.1.1. Dataset
          2. 4.2.1.2. Clustering
          3. 4.2.1.3. Optimum cluster labeling
          4. 4.2.1.4. Classification
      3. 4.3. Optimum cluster labeling
        1. 4.3.1. Problem definition
        2. 4.3.2. The Hungarian algorithm
          1. 4.3.2.1. Weighted complete bipartite graph
          2. 4.3.2.2. Matching, perfect matching and maximum weight matching
          3. 4.3.2.3. Objective of Hungarian method
          4. 4.3.2.4. Complexity considerations
        3. 4.3.3. Genetic algorithms
          1. 4.3.3.1. Reproduction operators
            1. 4.3.3.1.1. Crossover
            2. 4.3.3.1.2. Mutation
          2. 4.3.3.2. Forming the next generation
            1. 4.3.3.2.1. Generational replacement
            2. 4.3.3.2.2. Elitism with generational replacement
            3. 4.3.3.2.3. Steady state representation
          3. 4.3.3.3. GAs applied to optimum cluster labeling
          4. 4.3.3.4. Comparison of methods
      4. 4.4. Supervised classification block
        1. 4.4.1. Support vector machines
          1. 4.4.1.1. The kernel trick for nonlinearly separable classes
          2. 4.4.1.2. Multi-class classification
        2. 4.4.2. Example
      5. 4.5. Datasets
        1. 4.5.1. Mixtures of Gaussians
        2. 4.5.2. Datasets from the UCI repository
          1. 4.5.2.1. Iris dataset (Iris)
          2. 4.5.2.2. Wine dataset (wine)
          3. 4.5.2.3. Wisconsin breast cancer dataset (breast)
          4. 4.5.2.4. Handwritten digits dataset (Pendig)
          5. 4.5.2.5. Pima Indians diabetes (diabetes)
        3. 4.5.3. Utterance dataset
      6. 4.6. An analysis of the bounds for the cluster and label approaches
      7. 4.7. Extension through cluster pruning
        1. 4.7.1. Determination of silhouette thresholds
        2. 4.7.2. Evaluation of the cluster pruning approach
      8. 4.8. Simulations and results
      9. 4.9. Summary
  6. Part 3: Contributions to Unsupervised Classification — Algorithms to Detect the Optimal Number of Clusters
    1. Chapter 5: Detection of the Number of Clusters through Non-Parametric Clustering Algorithms
      1. 5.1. Introduction
      2. 5.2. New hierarchical pole-based clustering algorithm
        1. 5.2.1. Pole-based clustering basis module
        2. 5.2.2. Hierarchical pole-based clustering
      3. 5.3. Evaluation
        1. 5.3.1. Cluster evaluation metrics
      4. 5.4. Datasets
        1. 5.4.1. Results
        2. 5.4.2. Complexity considerations for large databases
      5. 5.5. Summary
    2. Chapter 6: Detecting the Number of Clusters through Cluster Validation
      1. 6.1. Introduction
      2. 6.2. Cluster validation methods
        1. 6.2.1. Dunn index
        2. 6.2.2. Hartigan
        3. 6.2.3. Davies Bouldin index
        4. 6.2.4. Krzanowski and Lai index
        5. 6.2.5. Silhouette
        6. 6.2.6. Hubert’s γ
        7. 6.2.7. Gap statistic
      3. 6.3. Combination approach based on quantiles
      4. 6.4. Datasets
        1. 6.4.1. Mixtures of Gaussians
        2. 6.4.2. Cancer DNA-microarray dataset
        3. 6.4.3. Iris dataset
      5. 6.5. Results
        1. 6.5.1. Validation results of the five Gaussian dataset
        2. 6.5.2. Validation results of the mixture of seven Gaussians
        3. 6.5.3. Validation results of the NCI60 dataset
        4. 6.5.4. Validation results of the Iris dataset
        5. 6.5.5. Discussion
      6. 6.6. Application of speech utterances
      7. 6.7. Summary
  7. Bibliography
  8. Index

Product information

  • Title: Semi-Supervised and Unsupervised Machine Learning: Novel Strategies
  • Author(s): Amparo Albalate, Wolfgang Minker
  • Release date: January 2011
  • Publisher(s): Wiley
  • ISBN: 9781848212039