book

Python Machine Learning

Name: Python Machine Learning
Author: Sebastian Raschka
ISBN: 9781783555130

by Sebastian Raschka

September 2015

Beginner to intermediate

454 pages

10h 49m

English

Packt Publishing

Read now

Unlock full access

Python Machine Learning
Table of Contents
Python Machine Learning
Credits
Foreword
About the Author
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers, and moreWhy subscribe?Free access for Packt account holders
Preface
What this book covers
What you need for this book

Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions
1. Giving Computers the Ability to Learn from Data
Building intelligent machines to transform data into knowledge
The three different types of machine learning
Making predictions about the future with supervised learningClassification for predicting class labelsRegression for predicting continuous outcomesSolving interactive problems with reinforcement learningDiscovering hidden structures with unsupervised learningFinding subgroups with clusteringDimensionality reduction for data compression
An introduction to the basic terminology and notations
A roadmap for building machine learning systems
Preprocessing – getting data into shapeTraining and selecting a predictive modelEvaluating models and predicting unseen data instances
Using Python for machine learning
Installing Python packages
Summary
2. Training Machine Learning Algorithms for Classification
Artificial neurons – a brief glimpse into the early history of machine learning
Implementing a perceptron learning algorithm in Python
Training a perceptron model on the Iris dataset
Adaptive linear neurons and the convergence of learning
Minimizing cost functions with gradient descentImplementing an Adaptive Linear Neuron in PythonLarge scale machine learning and stochastic gradient descent
Summary
3. A Tour of Machine Learning Classifiers Using Scikit-learn
Choosing a classification algorithm
First steps with scikit-learn
Training a perceptron via scikit-learn
Modeling class probabilities via logistic regression
Logistic regression intuition and conditional probabilitiesLearning the weights of the logistic cost functionTraining a logistic regression model with scikit-learnTackling overfitting via regularization
Maximum margin classification with support vector machines
Maximum margin intuitionDealing with the nonlinearly separable case using slack variablesAlternative implementations in scikit-learn
Solving nonlinear problems using a kernel SVM
Using the kernel trick to find separating hyperplanes in higher dimensional space
Decision tree learning
Maximizing information gain – getting the most bang for the buckBuilding a decision treeCombining weak to strong learners via random forests
K-nearest neighbors – a lazy learning algorithm
Summary
4. Building Good Training Sets – Data Preprocessing
Dealing with missing dataEliminating samples or features with missing valuesImputing missing valuesUnderstanding the scikit-learn estimator API
Handling categorical data
Mapping ordinal featuresEncoding class labelsPerforming one-hot encoding on nominal features
Partitioning a dataset in training and test sets
Bringing features onto the same scale
Selecting meaningful features
Sparse solutions with L1 regularizationSequential feature selection algorithms
Assessing feature importance with random forests
Summary
5. Compressing Data via Dimensionality Reduction
Unsupervised dimensionality reduction via principal component analysisTotal and explained varianceFeature transformationPrincipal component analysis in scikit-learn
Supervised data compression via linear discriminant analysis
Computing the scatter matricesSelecting linear discriminants for the new feature subspaceProjecting samples onto the new feature spaceLDA via scikit-learn
Using kernel principal component analysis for nonlinear mappings
Kernel functions and the kernel trickImplementing a kernel principal component analysis in PythonExample 1 – separating half-moon shapesExample 2 – separating concentric circlesProjecting new data pointsKernel principal component analysis in scikit-learn
Summary
6. Learning Best Practices for Model Evaluation and Hyperparameter Tuning
Streamlining workflows with pipelinesLoading the Breast Cancer Wisconsin datasetCombining transformers and estimators in a pipeline
Using k-fold cross-validation to assess model performance
The holdout methodK-fold cross-validation
Debugging algorithms with learning and validation curves
Diagnosing bias and variance problems with learning curvesAddressing overfitting and underfitting with validation curves
Fine-tuning machine learning models via grid search
Tuning hyperparameters via grid searchAlgorithm selection with nested cross-validation
Looking at different performance evaluation metrics
Reading a confusion matrixOptimizing the precision and recall of a classification modelPlotting a receiver operating characteristicThe scoring metrics for multiclass classification
Summary
7. Combining Different Models for Ensemble Learning
Learning with ensembles
Implementing a simple majority vote classifier
Combining different algorithms for classification with majority vote
Evaluating and tuning the ensemble classifier
Bagging – building an ensemble of classifiers from bootstrap samples
Leveraging weak learners via adaptive boosting
Summary
8. Applying Machine Learning to Sentiment Analysis
Obtaining the IMDb movie review dataset
Introducing the bag-of-words model
Transforming words into feature vectorsAssessing word relevancy via term frequency-inverse document frequencyCleaning text dataProcessing documents into tokens
Training a logistic regression model for document classification
Working with bigger data – online algorithms and out-of-core learning
Summary
9. Embedding a Machine Learning Model into a Web Application
Serializing fitted scikit-learn estimators
Setting up a SQLite database for data storage
Developing a web application with Flask
Our first Flask web applicationForm validation and rendering
Turning the movie classifier into a web application
Deploying the web application to a public server
Updating the movie review classifier
Summary
10. Predicting Continuous Target Variables with Regression Analysis
Introducing a simple linear regression model
Exploring the Housing Dataset
Visualizing the important characteristics of a dataset
Implementing an ordinary least squares linear regression model
Solving regression for regression parameters with gradient descentEstimating the coefficient of a regression model via scikit-learn
Fitting a robust regression model using RANSAC
Evaluating the performance of linear regression models
Using regularized methods for regression
Turning a linear regression model into a curve – polynomial regression
Modeling nonlinear relationships in the Housing DatasetDealing with nonlinear relationships using random forestsDecision tree regressionRandom forest regression
Summary
11. Working with Unlabeled Data – Clustering Analysis
Grouping objects by similarity using k-meansK-means++Hard versus soft clusteringUsing the elbow method to find the optimal number of clustersQuantifying the quality of clustering via silhouette plots
Organizing clusters as a hierarchical tree
Performing hierarchical clustering on a distance matrixAttaching dendrograms to a heat mapApplying agglomerative clustering via scikit-learn
Locating regions of high density via DBSCAN
Summary
12. Training Artificial Neural Networks for Image Recognition
Modeling complex functions with artificial neural networksSingle-layer neural network recapIntroducing the multi-layer neural network architectureActivating a neural network via forward propagation
Classifying handwritten digits
Obtaining the MNIST datasetImplementing a multi-layer perceptron
Training an artificial neural network
Computing the logistic cost functionTraining neural networks via backpropagation
Developing your intuition for backpropagation
Debugging neural networks with gradient checking
Convergence in neural networks
Other neural network architectures
Convolutional Neural NetworksRecurrent Neural Networks
A few last words about neural network implementation
Summary
13. Parallelizing Neural Network Training with Theano
Building, compiling, and running expressions with TheanoWhat is Theano?First steps with TheanoConfiguring TheanoWorking with array structuresWrapping things up – a linear regression example
Choosing activation functions for feedforward neural networks
Logistic function recapEstimating probabilities in multi-class classification via the softmax functionBroadening the output spectrum by using a hyperbolic tangent
Training neural networks efficiently using Keras
Summary
Index

Content preview from Python Machine Learning

Using regularized methods for regression

As we discussed in Chapter 3, A Tour of Machine Learning Classifiers Using Scikit-learn, regularization is one approach to tackle the problem of overfitting by adding additional information, and thereby shrinking the parameter values of the model to induce a penalty against complexity. The most popular approaches to regularized linear regression are the so-called Ridge Regression, Least Absolute Shrinkage and Selection Operator (LASSO) and Elastic Net method.

Ridge regression is an L2 penalized model where we simply add the squared sum of the weights to our least-squares cost function:

Here:

By increasing ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781783555130

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Python Machine Learning

by Sebastian Raschka

Using regularized methods for regression

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.