book

Automated Machine Learning in Action

Name: Automated Machine Learning in Action
ISBN: 9781617298059

by Qingquan Song, Xia Hu, Haifeng Jin

May 2022

Intermediate to advanced

336 pages

10h 7m

English

Manning Publications

Read now

Unlock full access

inside front cover
title
Copyright
contents
front matter
prefaceacknowledgmentsabout this bookWho should read this bookHow this book is organized: A road mapAbout the codeliveBook discussion forumOther online resourcesabout the authorsabout the cover illustration
Part 1 Fundamentals of AutoML
1 From machine learning to automated machine learning
1.1 A glimpse of automated machine learning1.2 Getting started with machine learning1.2.1 What is machine learning?1.2.2 The machine learning process1.2.3 Hyperparameter tuning1.2.4 The obstacles to applying machine learning1.3 AutoML: The automation of automation1.3.1 Three key components of AutoML1.3.2 Are we able to achieve full automation?Summary
2 The end-to-end pipeline of an ML project
2.1 An overview of the end-to-end pipeline2.2 Framing the problem and assembling the dataset2.3 Data preprocessing2.4 Feature engineering2.5 ML algorithm selection2.5.1 Building the linear regression model2.5.2 Building the decision tree model2.6 Fine-tuning the ML model: Introduction to grid searchSummary
3 Deep learning in a nutshell
3.1 What is deep learning?3.2 TensorFlow and Keras3.3 California housing price prediction with a multilayer perceptron3.3.1 Assembling and preparing the data3.3.2 Building up the multilayer perceptron3.3.3 Training and testing the neural network3.3.4 Tuning the number of epochs3.4 Classifying handwritten digits with convolutional neural networks3.4.1 Assembling and preparing the dataset3.4.2 Addressing the problem with an MLP3.4.3 Addressing the problem with a CNN3.5 IMDB review classification with recurrent neural networks3.5.1 Preparing the data3.5.2 Building up the RNN3.5.3 Training and validating the RNNSummary
Part 2 AutoML in practice

4 Automated generation of end-to-end ML solutions
4.1 Preparing the AutoML toolkit: AutoKeras4.2 Automated image classification4.2.1 Attacking the problem with five lines of code4.2.2 Dealing with different data formats4.2.3 Configuring the tuning process4.3 End-to-end AutoML solutions for four supervised learning problems4.3.1 Text classification with the 20 newsgroups dataset4.3.2 Structured data classification with the Titanic dataset4.3.3 Structured data regression with the California housing dataset4.3.4 Multilabel image classification4.4 Addressing tasks with multiple inputs or outputs4.4.1 Automated image classification with the AutoKeras IO API4.4.2 Automated multi-input learning4.4.3 Automated multi-output learningSummary
5 Customizing the search space by creating AutoML pipelines
5.1 Working with sequential AutoML pipelines5.2 Creating a sequential AutoML pipeline for automated hyperparameter tuning5.2.1 Tuning MLPs for structured data regression5.2.2 Tuning CNNs for image classification5.3 Automated pipeline search with hyperblocks5.3.1 Automated model selection for image classification5.3.2 Automated selection of image preprocessing methods5.4 Designing a graph-structured AutoML pipeline5.5 Designing custom AutoML blocks5.5.1 Tuning MLPs with a custom MLP block5.5.2 Designing a hyperblock for model selectionSummary
6 AutoML with a fully customized search space
6.1 Customizing the search space in a layerwise fashion6.1.1 Tuning an MLP for regression with KerasTuner6.1.2 Tuning an autoencoder model for unsupervised learning6.2 Tuning the autoencoder model6.3 Tuning shallow models with different search methods6.3.1 Selecting and tuning shallow models6.3.2 Tuning a shallow model pipeline6.3.3 Trying out different search methods6.3.4 Automated feature engineering6.4 Controlling the AutoML process by customizing tuners6.4.1 Creating a tuner for tuning scikit-learn models6.4.2 Creating a tuner for tuning Keras models6.4.3 Jointly tuning and selection among deep learning and shallow models6.4.4 Hyperparameter tuning beyond Keras and scikit-learn modelsSummary
Part 3 Advanced topics in AutoML
7 Customizing the search method of AutoML
7.1 Sequential search methods7.2 Getting started with a random search method7.3 Customizing a Bayesian optimization search method7.3.1 Vectorizing the hyperparameters7.3.2 Updating the surrogate function based on historical model evaluations7.3.3 Designing the acquisition function7.3.4 Sampling the new hyperparameters via the acquisition function7.3.5 Tuning the GBDT model with the Bayesian optimization method7.3.6 Resuming the search process and recovering the search method7.4 Customizing an evolutionary search method7.4.1 Selection strategies in the evolutionary search method7.4.2 The aging evolutionary search method7.4.3 Implementing a simple mutation operation7.4.4 Evaluating the aging evolutionary search methodSummary
8 Scaling up AutoML
8.1 Handling large-scale datasets8.1.1 Loading an image-classification dataset8.1.2 Splitting the loaded dataset8.1.3 Loading a text-classification dataset8.1.4 Handling large datasets in general8.2 Parallelization on multiple GPUs8.2.1 Data parallelism8.2.2 Model parallelism8.2.3 Parallel tuning8.3 Search speedup strategies8.3.1 Model scheduling with Hyperband8.3.2 Faster convergence with pretrained weights in the search space8.3.3 Warm-starting the search spaceSummary
9 Wrapping up
9.1 Key concepts in review9.1.1 The AutoML process and its key components9.1.2 The machine learning pipeline9.1.3 The taxonomy of AutoML9.1.4 Applications of AutoML9.1.5 Automated deep learning with AutoKeras9.1.6 Fully personalized AutoML with KerasTuner9.1.7 Implementing search techniques9.1.8 Scaling up the AutoML process9.2 AutoML tools and platforms9.2.1 Open source AutoML tools9.2.2 Commercial AutoML platforms9.3 The challenges and future of AutoML9.3.1 Measuring the performance of AutoML9.3.2 Resource complexity9.3.3 Interpretability and transparency9.3.4 Reproducibility and robustness9.3.5 Generalizability and transferability9.3.6 Democratization and productionization9.4 Staying up-to-date in a fast-moving fieldSummary
Appendix A. Setting up an environment for running code
A.1 Getting started with Google ColaboratoryA.1.1 Basic Google Colab notebook operationsA.1.2 Packages and hardware configurationA.2 Setting up a Jupyter Notebook environment on a local Ubuntu systemA.2.1 Creating a Python 3 virtual environmentA.2.2 Installing the required Python packagesA.2.3 Setting up the IPython kernelA.2.4 Working on the Jupyter notebooks
Appendix B. Three examples: Classification of image, text, and tabular data
B.1 Image classification: Recognizing handwritten digitsB.1.1 Problem framing and data assemblyB.1.2 Exploring and preparing the dataB.1.3 Using principal component analysis to condense the featuresB.1.4 Classification with a support vector machineB.1.5 Building a data-processing pipeline with PCA and SVMPCA (principal component analysis)B.1.6 Jointly tuning multiple components in the pipelineB.2 Text classification: Classifying topics of newsgroup postsB.2.1 Problem framing and data assemblyB.2.2 Data preprocessing and feature engineeringB.2.3 Building a text classifier with the logistic regression modelB.2.4 Building a text classifier with the naive Bayes modelB.2.5 Tuning the text-classification pipeline with grid searchB.3 Tabular classification: Identifying Titanic survivorsB.3.1 Problem framing and data assemblyB.3.2 Data preprocessing and feature engineeringB.3.3 Building tree-based classifiers
index
inside back cover

Content preview from Automated Machine Learning in Action

8 Scaling up AutoML

This chapter covers

Loading large datasets into memory batch by batch
Using multiple GPUs to speed up search and training
Using Hyperband to efficiently schedule model training to make the best use of the available computing resources
Using pretrained models and warm-start to accelerate the search process

This chapter introduces various techniques for large-scale training—for example, using large datasets to train large models on multiple GPUs. For datasets that are too big to fit into memory all at once, we’ll show you how to load them batch by batch during the training. We’ll also introduce different parallelization strategies to distribute the training and search processes onto multiple GPUs. In addition, we’ll show you ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781617298059Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Automated Machine Learning in Action

by Qingquan Song, Xia Hu, Haifeng Jin

8 Scaling up AutoML

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.