book

Python: Deeper Insights into Machine Learning

Name: Python: Deeper Insights into Machine Learning
ISBN: 9781787128576

by Sebastian Raschka, David Julian, John Hearty

August 2016

Beginner to intermediate

901 pages

22h 30m

English

Packt Publishing

Read now

Unlock full access

Python: Deeper Insights into Machine Learning
Table of Contents
Python: Deeper Insights into Machine Learning
Python: Deeper Insights into Machine Learning
Credits
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions

1. Module 1
1. Giving Computers the Ability to Learn from Data
Building intelligent machines to transform data into knowledge
The three different types of machine learning
Making predictions about the future with supervised learningClassification for predicting class labelsRegression for predicting continuous outcomesSolving interactive problems with reinforcement learningDiscovering hidden structures with unsupervised learningFinding subgroups with clusteringDimensionality reduction for data compression
An introduction to the basic terminology and notations
A roadmap for building machine learning systems
Preprocessing – getting data into shapeTraining and selecting a predictive modelEvaluating models and predicting unseen data instances
Using Python for machine learning
Installing Python packages
Summary
2. Training Machine Learning Algorithms for Classification
Artificial neurons – a brief glimpse into the early history of machine learning
Implementing a perceptron learning algorithm in Python
Training a perceptron model on the Iris dataset
Adaptive linear neurons and the convergence of learning
Minimizing cost functions with gradient descentImplementing an Adaptive Linear Neuron in PythonLarge scale machine learning and stochastic gradient descent
Summary
3. A Tour of Machine Learning Classifiers Using Scikit-learn
Choosing a classification algorithm
First steps with scikit-learn
Training a perceptron via scikit-learn
Modeling class probabilities via logistic regression
Logistic regression intuition and conditional probabilitiesLearning the weights of the logistic cost functionTraining a logistic regression model with scikit-learnTackling overfitting via regularization
Maximum margin classification with support vector machines
Maximum margin intuitionDealing with the nonlinearly separable case using slack variablesAlternative implementations in scikit-learn
Solving nonlinear problems using a kernel SVM
Using the kernel trick to find separating hyperplanes in higher dimensional space
Decision tree learning
Maximizing information gain – getting the most bang for the buckBuilding a decision treeCombining weak to strong learners via random forests
K-nearest neighbors – a lazy learning algorithm
Summary
4. Building Good Training Sets – Data Preprocessing
Dealing with missing dataEliminating samples or features with missing valuesImputing missing valuesUnderstanding the scikit-learn estimator API
Handling categorical data
Mapping ordinal featuresEncoding class labelsPerforming one-hot encoding on nominal features
Partitioning a dataset in training and test sets
Bringing features onto the same scale
Selecting meaningful features
Sparse solutions with L1 regularizationSequential feature selection algorithms
Assessing feature importance with random forests
Summary
5. Compressing Data via Dimensionality Reduction
Unsupervised dimensionality reduction via principal component analysisTotal and explained varianceFeature transformationPrincipal component analysis in scikit-learn
Supervised data compression via linear discriminant analysis
Computing the scatter matricesSelecting linear discriminants for the new feature subspaceProjecting samples onto the new feature spaceLDA via scikit-learn
Using kernel principal component analysis for nonlinear mappings
Kernel functions and the kernel trickImplementing a kernel principal component analysis in PythonExample 1 – separating half-moon shapesExample 2 – separating concentric circlesProjecting new data pointsKernel principal component analysis in scikit-learn
Summary
6. Learning Best Practices for Model Evaluation and Hyperparameter Tuning
Streamlining workflows with pipelinesLoading the Breast Cancer Wisconsin datasetCombining transformers and estimators in a pipeline
Using k-fold cross-validation to assess model performance
The holdout methodK-fold cross-validation
Debugging algorithms with learning and validation curves
Diagnosing bias and variance problems with learning curvesAddressing overfitting and underfitting with validation curves
Fine-tuning machine learning models via grid search
Tuning hyperparameters via grid searchAlgorithm selection with nested cross-validation
Looking at different performance evaluation metrics
Reading a confusion matrixOptimizing the precision and recall of a classification modelPlotting a receiver operating characteristicThe scoring metrics for multiclass classification
Summary
7. Combining Different Models for Ensemble Learning
Learning with ensembles
Implementing a simple majority vote classifier
Combining different algorithms for classification with majority vote
Evaluating and tuning the ensemble classifier
Bagging – building an ensemble of classifiers from bootstrap samples
Leveraging weak learners via adaptive boosting
Summary
8. Applying Machine Learning to Sentiment Analysis
Obtaining the IMDb movie review dataset
Introducing the bag-of-words model
Transforming words into feature vectorsAssessing word relevancy via term frequency-inverse document frequencyCleaning text dataProcessing documents into tokens
Training a logistic regression model for document classification
Working with bigger data – online algorithms and out-of-core learning
Summary
9. Embedding a Machine Learning Model into a Web Application
Serializing fitted scikit-learn estimators
Setting up a SQLite database for data storage
Developing a web application with Flask
Our first Flask web applicationForm validation and rendering
Turning the movie classifier into a web application
Deploying the web application to a public server
Updating the movie review classifier
Summary
10. Predicting Continuous Target Variables with Regression Analysis
Introducing a simple linear regression model
Exploring the Housing Dataset
Visualizing the important characteristics of a dataset
Implementing an ordinary least squares linear regression model
Solving regression for regression parameters with gradient descentEstimating the coefficient of a regression model via scikit-learn
Fitting a robust regression model using RANSAC
Evaluating the performance of linear regression models
Using regularized methods for regression
Turning a linear regression model into a curve – polynomial regression
Modeling nonlinear relationships in the Housing DatasetDealing with nonlinear relationships using random forestsDecision tree regressionRandom forest regression
Summary
11. Working with Unlabeled Data – Clustering Analysis
Grouping objects by similarity using k-meansK-means++Hard versus soft clusteringUsing the elbow method to find the optimal number of clustersQuantifying the quality of clustering via silhouette plots
Organizing clusters as a hierarchical tree
Performing hierarchical clustering on a distance matrixAttaching dendrograms to a heat mapApplying agglomerative clustering via scikit-learn
Locating regions of high density via DBSCAN
Summary
12. Training Artificial Neural Networks for Image Recognition
Modeling complex functions with artificial neural networksSingle-layer neural network recapIntroducing the multi-layer neural network architectureActivating a neural network via forward propagation
Classifying handwritten digits
Obtaining the MNIST datasetImplementing a multi-layer perceptron
Training an artificial neural network
Computing the logistic cost functionTraining neural networks via backpropagation
Developing your intuition for backpropagation
Debugging neural networks with gradient checking
Convergence in neural networks
Other neural network architectures
Convolutional Neural NetworksRecurrent Neural Networks
A few last words about neural network implementation
Summary
13. Parallelizing Neural Network Training with Theano
Building, compiling, and running expressions with TheanoWhat is Theano?First steps with TheanoConfiguring TheanoWorking with array structuresWrapping things up – a linear regression example
Choosing activation functions for feedforward neural networks
Logistic function recapEstimating probabilities in multi-class classification via the softmax functionBroadening the output spectrum by using a hyperbolic tangent
Training neural networks efficiently using Keras
Summary
2. Module 2
1. Thinking in Machine Learning
The human interface
Design principles
Types of questionsAre you asking the right question?TasksClassificationRegressionClusteringDimensionality reductionErrorsOptimizationLinear programmingModelsGeometric modelsProbabilistic modelsLogical modelsFeaturesUnified modeling languageClass diagramsObject diagramsActivity diagramsState diagrams
Summary
2. Tools and Techniques
Python for machine learning
IPython console
Installing the SciPy stack
NumPY
Constructing and transforming arraysMathematical operations
Matplotlib
Pandas
SciPy
Scikit-learn
Summary
3. Turning Data into Information
What is data?
Big data
Challenges of big dataData volumeData velocityData varietyData modelsData distributionsData from databasesData from the WebData from natural languageData from imagesData from application programming interfaces
Signals
Data from sound
Cleaning data
Visualizing data
Summary
4. Models – Learning from Information
Logical modelsGenerality orderingVersion spaceCoverage spacePAC learning and computational complexity
Tree models
Purity
Rule models
The ordered list approachSet-based rule models
Summary
5. Linear Models
Introducing least squaresGradient descentThe normal equation
Logistic regression
The Cost function for logistic regression
Multiclass classification
Regularization
Summary
6. Neural Networks
Getting started with neural networks
Logistic units
Cost function
Minimizing the cost function
Implementing a neural network
Gradient checking
Other neural net architectures
Summary
7. Features – How Algorithms See the World
Feature typesQuantitative featuresOrdinal featuresCategorical features
Operations and statistics
Structured features
Transforming features
DiscretizationNormalizationCalibration
Principle component analysis
Summary
8. Learning with Ensembles
Ensemble types
Bagging
Random forestsExtra trees
Boosting
AdaboostGradient boosting
Ensemble strategies
Other methods
Summary
9. Design Strategies and Case Studies
Evaluating model performance
Model selection
Gridsearch
Learning curves
Real-world case studies
Building a recommender systemContent-based filteringCollaborative filteringReviewing the case studyInsect detection in greenhousesReviewing the case study
Machine learning at a glance
Summary
3. Module 3
1. Unsupervised Machine Learning
Principal component analysisPCA – a primerEmploying PCA
Introducing k-means clustering
Clustering – a primerKick-starting clustering analysisTuning your clustering configurations
Self-organizing maps
SOM – a primerEmploying SOM
Further reading
Summary
2. Deep Belief Networks
Neural networks – a primerThe composition of a neural networkNetwork topologies
Restricted Boltzmann Machine
Introducing the RBMTopologyTrainingApplications of the RBMFurther applications of the RBM
Deep belief networks
Training a DBNApplying the DBNValidating the DBN
Further reading
Summary
3. Stacked Denoising Autoencoders
AutoencodersIntroducing the autoencoderTopologyTrainingDenoising autoencodersApplying a dA
Stacked Denoising Autoencoders
Applying the SdAAssessing SdA performance
Further reading
Summary
4. Convolutional Neural Networks
Introducing the CNNUnderstanding the convnet topologyUnderstanding convolution layersUnderstanding pooling layersTraining a convnetPutting it all togetherApplying a CNN
Further Reading
Summary
5. Semi-Supervised Learning
Introduction
Understanding semi-supervised learning
Semi-supervised algorithms in action
Self-trainingImplementing self-trainingFinessing your self-training implementationImproving the selection processContrastive Pessimistic Likelihood Estimation
Further reading
Summary
6. Text Feature Engineering
Introduction
Text feature engineering
Cleaning text dataText cleaning with BeautifulSoupManaging punctuation and tokenizingTagging and categorising wordsTagging with NLTKSequential taggingBackoff taggingCreating features from text dataStemmingBagging and random forestsTesting our prepared data
Further reading
Summary
7. Feature Engineering Part II
Introduction
Creating a feature set
Engineering features for ML applicationsUsing rescaling techniques to improve the learnability of featuresCreating effective derived variablesReinterpreting non-numeric featuresUsing feature selection techniquesPerforming feature selectionCorrelationLASSORecursive Feature EliminationGenetic models
Feature engineering in practice
Acquiring data via RESTful APIsTesting the performance of our modelTwitterTranslink TwitterConsumer commentsThe Bing Traffic APIDeriving and selecting variables using feature engineering techniquesThe weather API
Further reading
Summary
8. Ensemble Methods
Introducing ensemblesUnderstanding averaging ensemblesUsing bagging algorithmsUsing random forestsApplying boosting methodsUsing XGBoostUsing stacking ensemblesApplying ensembles in practice
Using models in dynamic applications
Understanding model robustnessIdentifying modeling risk factorsStrategies to managing model robustness
Further reading
Summary
9. Additional Python Machine Learning Tools
Alternative development toolsIntroduction to LasagneGetting to know LasagneIntroduction to TensorFlowGetting to know TensorFlowUsing TensorFlow to iteratively improve our modelsKnowing when to use these libraries
Further reading
Summary
10. Chapter Code Requirements
A. Biblography
Index

Content preview from Python: Deeper Insights into Machine Learning

Chapter 8. Learning with Ensembles

The motivation for creating machine learning ensembles comes from clear intuitions and is grounded in a rich theoretical history. Diversity, in many natural and human-made systems, makes them more resilient to perturbations. Similarly, we have seen that averaging results from a number of measurements can often result in a more stable models that are less susceptible to random fluctuations, such as outliers or errors in data collection.

In this chapter, we will divide this rather large and diverse space into the following topics:

Ensemble types
Bagging
Random forests
Boosting

Ensemble types

Ensemble techniques can be broadly divided into two types:

Averaging method: This is the method in which several estimators are run ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781787128576

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills