O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Thoughtful Machine Learning with Python

Thoughtful Machine Learning with Python

by Matthew Kirk
Publisher: O'Reilly Media, Inc.
Release Date: January 2017
ISBN: 9781491924129
View table of contents
Start reading

Book Description

Gain the confidence you need to apply machine learning in your daily work. With this practical guide, author Matthew Kirk shows you how to integrate and test machine learning algorithms in your code, without the academic subtext.

Featuring graphs and highlighted code examples throughout, the book features tests with Python’s Numpy, Pandas, Scikit-Learn, and SciPy data science libraries. If you’re a software engineer or business analyst interested in data science, this book will help you:

  • Reference real-world examples to test each algorithm through engaging, hands-on exercises
  • Apply test-driven development (TDD) to write and run tests before you start coding
  • Explore techniques for improving your machine-learning models with data extraction and feature development
  • Watch out for the risks of machine learning, such as underfitting or overfitting data
  • Work with K-Nearest Neighbors, neural networks, clustering, and other algorithms

Table of Contents

  1. Preface
    1. Conventions Used in This Book
    2. Using Code Examples
    3. O’Reilly Safari
    4. How to Contact Us
    5. Acknowledgments
  2. 1. Probably Approximately Correct Software
    1. Writing Software Right
      1. SOLID
      2. Testing or TDD
      3. Refactoring
    2. Writing the Right Software
      1. Writing the Right Software with Machine Learning
      2. What Exactly Is Machine Learning?
      3. The High Interest Credit Card Debt of Machine Learning
      4. SOLID Applied to Machine Learning
      5. Machine Learning Code Is Complex but Not Impossible
      6. TDD: Scientific Method 2.0
      7. Refactoring Our Way to Knowledge
    3. The Plan for the Book
  3. 2. A Quick Introduction to Machine Learning
    1. What Is Machine Learning?
    2. Supervised Learning
    3. Unsupervised Learning
    4. Reinforcement Learning
    5. What Can Machine Learning Accomplish?
    6. Mathematical Notation Used Throughout the Book
    7. Conclusion
  4. 3. K-Nearest Neighbors
    1. How Do You Determine Whether You Want to Buy a House?
    2. How Valuable Is That House?
    3. Hedonic Regression
    4. What Is a Neighborhood?
    5. K-Nearest Neighbors
    6. Mr. K’s Nearest Neighborhood
    7. Distances
      1. Triangle Inequality
      2. Geometrical Distance
      3. Computational Distances
      4. Statistical Distances
    8. Curse of Dimensionality
    9. How Do We Pick K?
      1. Guessing K
      2. Heuristics for Picking K
    10. Valuing Houses in Seattle
      1. About the Data
      2. General Strategy
      3. Coding and Testing Design
      4. KNN Regressor Construction
      5. KNN Testing
    11. Conclusion
  5. 4. Naive Bayesian Classification
    1. Using Bayes’ Theorem to Find Fraudulent Orders
    2. Conditional Probabilities
    3. Probability Symbols
    4. Inverse Conditional Probability (aka Bayes’ Theorem)
    5. Naive Bayesian Classifier
      1. The Chain Rule
    6. Naiveté in Bayesian Reasoning
    7. Pseudocount
    8. Spam Filter
      1. Setup Notes
      2. Coding and Testing Design
      3. Data Source
      4. EmailObject
      5. Tokenization and Context
      6. SpamTrainer
      7. Error Minimization Through Cross-Validation
    9. Conclusion
  6. 5. Decision Trees and Random Forests
    1. The Nuances of Mushrooms
    2. Classifying Mushrooms Using a Folk Theorem
    3. Finding an Optimal Switch Point
      1. Information Gain
      2. GINI Impurity
      3. Variance Reduction
    4. Pruning Trees
      1. Ensemble Learning
      2. Writing a Mushroom Classifier
    5. Conclusion
  7. 6. Hidden Markov Models
    1. Tracking User Behavior Using State Machines
    2. Emissions/Observations of Underlying States
    3. Simplification Through the Markov Assumption
      1. Using Markov Chains Instead of a Finite State Machine
    4. Hidden Markov Model
    5. Evaluation: Forward-Backward Algorithm
      1. Mathematical Representation of the Forward-Backward Algorithm
      2. Using User Behavior
    6. The Decoding Problem Through the Viterbi Algorithm
    7. The Learning Problem
    8. Part-of-Speech Tagging with the Brown Corpus
      1. Setup Notes
      2. Coding and Testing Design
      3. The Seam of Our Part-of-Speech Tagger: CorpusParser
      4. Writing the Part-of-Speech Tagger
      5. Cross-Validating to Get Confidence in the Model
      6. How to Make This Model Better
    9. Conclusion
  8. 7. Support Vector Machines
    1. Customer Happiness as a Function of What They Say
      1. Sentiment Classification Using SVMs
    2. The Theory Behind SVMs
      1. Decision Boundary
      2. Maximizing Boundaries
      3. Kernel Trick: Feature Transformation
      4. Optimizing with Slack
    3. Sentiment Analyzer
      1. Setup Notes
      2. Coding and Testing Design
      3. SVM Testing Strategies
      4. Corpus Class
      5. CorpusSet Class
      6. Model Validation and the Sentiment Classifier
    4. Aggregating Sentiment
      1. Exponentially Weighted Moving Average
    5. Mapping Sentiment to Bottom Line
    6. Conclusion
  9. 8. Neural Networks
    1. What Is a Neural Network?
    2. History of Neural Nets
    3. Boolean Logic
    4. Perceptrons
    5. How to Construct Feed-Forward Neural Nets
      1. Input Layer
      2. Hidden Layers
      3. Neurons
      4. Activation Functions
      5. Output Layer
      6. Training Algorithms
      7. The Delta Rule
      8. Back Propagation
      9. QuickProp
      10. RProp
    6. Building Neural Networks
      1. How Many Hidden Layers?
      2. How Many Neurons for Each Layer?
      3. Tolerance for Error and Max Epochs
    7. Using a Neural Network to Classify a Language
      1. Setup Notes
      2. Coding and Testing Design
      3. The Data
      4. Writing the Seam Test for Language
      5. Cross-Validating Our Way to a Network Class
      6. Tuning the Neural Network
      7. Precision and Recall for Neural Networks
      8. Wrap-Up of Example
      9. Conclusion
  10. 9. Clustering
    1. Studying Data Without Any Bias
    2. User Cohorts
    3. Testing Cluster Mappings
      1. Fitness of a Cluster
      2. Silhouette Coefficient
      3. Comparing Results to Ground Truth
    4. K-Means Clustering
      1. The K-Means Algorithm
      2. Downside of K-Means Clustering
    5. EM Clustering
      1. Algorithm
    6. The Impossibility Theorem
    7. Example: Categorizing Music
      1. Setup Notes
      2. Gathering the Data
      3. Coding Design
      4. Analyzing the Data with K-Means
      5. EM Clustering Our Data
      6. The Results from the EM Jazz Clustering
    8. Conclusion
  11. 10. Improving Models and Data Extraction
    1. Debate Club
    2. Picking Better Data
      1. Feature Selection
      2. Exhaustive Search
      3. Random Feature Selection
      4. A Better Feature Selection Algorithm
      5. Minimum Redundancy Maximum Relevance Feature Selection
    3. Feature Transformation and Matrix Factorization
      1. Principal Component Analysis
      2. Independent Component Analysis
    4. Ensemble Learning
      1. Bagging
      2. Boosting
    5. Conclusion
  12. 11. Putting It Together: Conclusion
    1. Machine Learning Algorithms Revisited
    2. How to Use This Information to Solve Problems
    3. What’s Next for You?
  13. Index