Thoughtful Machine Learning with Python

Table of contents

1. Preface
   1. Conventions Used in This Book
   2. Using Code Examples
   3. O’Reilly Safari
   4. How to Contact Us
   5. Acknowledgments
2. 1. Probably Approximately Correct Software
   1. Writing Software Right
      1. SOLID
      2. Testing or TDD
      3. Refactoring
   2. Writing the Right Software
      1. Writing the Right Software with Machine Learning
      2. What Exactly Is Machine Learning?
      3. The High Interest Credit Card Debt of Machine Learning
      4. SOLID Applied to Machine Learning
      5. Machine Learning Code Is Complex but Not Impossible
      6. TDD: Scientific Method 2.0
      7. Refactoring Our Way to Knowledge
   3. The Plan for the Book
3. 2. A Quick Introduction to Machine Learning
   1. What Is Machine Learning?
   2. Supervised Learning
   3. Unsupervised Learning
   4. Reinforcement Learning
   5. What Can Machine Learning Accomplish?
   6. Mathematical Notation Used Throughout the Book
   7. Conclusion
4. 3. K-Nearest Neighbors
   1. How Do You Determine Whether You Want to Buy a House?
   2. How Valuable Is That House?
   3. Hedonic Regression
   4. What Is a Neighborhood?
   5. K-Nearest Neighbors
   6. Mr. K’s Nearest Neighborhood
   7. Distances
      1. Triangle Inequality
      2. Geometrical Distance
      3. Computational Distances
      4. Statistical Distances
   8. Curse of Dimensionality
   9. How Do We Pick K?
      1. Guessing K
      2. Heuristics for Picking K
   10. Valuing Houses in Seattle
       1. About the Data
       2. General Strategy
       3. Coding and Testing Design
       4. KNN Regressor Construction
       5. KNN Testing
   11. Conclusion
5. 4. Naive Bayesian Classification
   1. Using Bayes’ Theorem to Find Fraudulent Orders
   2. Conditional Probabilities
   3. Probability Symbols
   4. Inverse Conditional Probability (aka Bayes’ Theorem)
   5. Naive Bayesian Classifier
      1. The Chain Rule
   6. Naiveté in Bayesian Reasoning
   7. Pseudocount
   8. Spam Filter
      1. Setup Notes
      2. Coding and Testing Design
      3. Data Source
      4. EmailObject
      5. Tokenization and Context
      6. SpamTrainer
      7. Error Minimization Through Cross-Validation
   9. Conclusion
6. 5. Decision Trees and Random Forests
   1. The Nuances of Mushrooms
   2. Classifying Mushrooms Using a Folk Theorem
   3. Finding an Optimal Switch Point
      1. Information Gain
      2. GINI Impurity
      3. Variance Reduction
   4. Pruning Trees
      1. Ensemble Learning
      2. Writing a Mushroom Classifier
   5. Conclusion
7. 6. Hidden Markov Models
   1. Tracking User Behavior Using State Machines
   2. Emissions/Observations of Underlying States
   3. Simplification Through the Markov Assumption
      1. Using Markov Chains Instead of a Finite State Machine
   4. Hidden Markov Model
   5. Evaluation: Forward-Backward Algorithm
      1. Mathematical Representation of the Forward-Backward Algorithm
      2. Using User Behavior
   6. The Decoding Problem Through the Viterbi Algorithm
   7. The Learning Problem
   8. Part-of-Speech Tagging with the Brown Corpus
      1. Setup Notes
      2. Coding and Testing Design
      3. The Seam of Our Part-of-Speech Tagger: CorpusParser
      4. Writing the Part-of-Speech Tagger
      5. Cross-Validating to Get Confidence in the Model
      6. How to Make This Model Better
   9. Conclusion
8. 7. Support Vector Machines
   1. Customer Happiness as a Function of What They Say
      1. Sentiment Classification Using SVMs
   2. The Theory Behind SVMs
      1. Decision Boundary
      2. Maximizing Boundaries
      3. Kernel Trick: Feature Transformation
      4. Optimizing with Slack
   3. Sentiment Analyzer
      1. Setup Notes
      2. Coding and Testing Design
      3. SVM Testing Strategies
      4. Corpus Class
      5. CorpusSet Class
      6. Model Validation and the Sentiment Classifier
   4. Aggregating Sentiment
      1. Exponentially Weighted Moving Average
   5. Mapping Sentiment to Bottom Line
   6. Conclusion
9. 8. Neural Networks
   1. What Is a Neural Network?
   2. History of Neural Nets
   3. Boolean Logic
   4. Perceptrons
   5. How to Construct Feed-Forward Neural Nets
      1. Input Layer
      2. Hidden Layers
      3. Neurons
      4. Activation Functions
      5. Output Layer
      6. Training Algorithms
      7. The Delta Rule
      8. Back Propagation
      9. QuickProp
      10. RProp
   6. Building Neural Networks
      1. How Many Hidden Layers?
      2. How Many Neurons for Each Layer?
      3. Tolerance for Error and Max Epochs
   7. Using a Neural Network to Classify a Language
      1. Setup Notes
      2. Coding and Testing Design
      3. The Data
      4. Writing the Seam Test for Language
      5. Cross-Validating Our Way to a Network Class
      6. Tuning the Neural Network
      7. Precision and Recall for Neural Networks
      8. Wrap-Up of Example
      9. Conclusion
10. 9. Clustering
    1. Studying Data Without Any Bias
    2. User Cohorts
    3. Testing Cluster Mappings
       1. Fitness of a Cluster
       2. Silhouette Coefficient
       3. Comparing Results to Ground Truth
    4. K-Means Clustering
       1. The K-Means Algorithm
       2. Downside of K-Means Clustering
    5. EM Clustering
       1. Algorithm
    6. The Impossibility Theorem
    7. Example: Categorizing Music
       1. Setup Notes
       2. Gathering the Data
       3. Coding Design
       4. Analyzing the Data with K-Means
       5. EM Clustering Our Data
       6. The Results from the EM Jazz Clustering
    8. Conclusion
11. 10. Improving Models and Data Extraction
    1. Debate Club
    2. Picking Better Data
       1. Feature Selection
       2. Exhaustive Search
       3. Random Feature Selection
       4. A Better Feature Selection Algorithm
       5. Minimum Redundancy Maximum Relevance Feature Selection
    3. Feature Transformation and Matrix Factorization
       1. Principal Component Analysis
       2. Independent Component Analysis
    4. Ensemble Learning
       1. Bagging
       2. Boosting
    5. Conclusion
12. 11. Putting It Together: Conclusion
    1. Machine Learning Algorithms Revisited
    2. How to Use This Information to Solve Problems
    3. What’s Next for You?
13. Index