book

The Deep Learning with Keras Workshop

by Matthew Moocarme, Mahla Abdolahnejad, Ritesh Bhagwat

July 2020

Intermediate to advanced

496 pages

9h 10m

English

Packt Publishing

Read now

Unlock full access

About the BookAudienceAbout the ChaptersConventionsCode PresentationSetting up Your EnvironmentInstalling AnacondaInstalling LibrariesRunning Jupyter Notebook Accessing the Code Files
IntroductionData RepresentationTables of DataLoading DataExercise 1.01: Loading a Dataset from the UCI Machine Learning RepositoryData PreprocessingExercise 1.02: Cleaning the DataAppropriate Representation of the DataExercise 1.03: Appropriate Representation of the DataLife Cycle of Model CreationMachine Learning Librariesscikit-learnKerasAdvantages of KerasDisadvantages of KerasMore Than Building ModelsModel TrainingClassifiers and Regression ModelsClassification TasksRegression TasksTraining Datasets and Test DatasetsModel Evaluation MetricsExercise 1.04: Creating a Simple ModelModel TuningBaseline ModelsExercise 1.05: Determining a Baseline ModelRegularizationCross-ValidationActivity 1.01: Adding Regularization to the ModelSummary
IntroductionAdvantages of ANNs over Traditional Machine Learning AlgorithmsAdvantages of Traditional Machine Learning Algorithms over ANNsHierarchical Data RepresentationLinear TransformationsScalars, Vectors, Matrices, and TensorsTensor AdditionExercise 2.01: Performing Various Operations with Vectors, Matrices, and TensorsReshapingMatrix TranspositionExercise 2.02: Matrix Reshaping and TranspositionMatrix MultiplicationExercise 2.03: Matrix MultiplicationExercise 2.04: Tensor MultiplicationIntroduction to KerasLayer TypesActivation FunctionsModel FittingActivity 2.01: Creating a Logistic Regression Model Using KerasSummary
IntroductionBuilding Your First Neural NetworkLogistic Regression to a Deep Neural NetworkActivation FunctionsForward Propagation for Making PredictionsLoss FunctionBackpropagation for Computing Derivatives of Loss FunctionGradient Descent for Learning ParametersExercise 3.01: Neural Network Implementation with KerasActivity 3.01: Building a Single-Layer Neural Network for Performing Binary ClassificationModel EvaluationEvaluating a Trained Model with KerasSplitting Data into Training and Test SetsUnderfitting and OverfittingEarly StoppingActivity 3.02: Advanced Fibrosis Diagnosis with Neural NetworksSummary
IntroductionCross-ValidationDrawbacks of Splitting a Dataset Only OnceK-Fold Cross-ValidationLeave-One-Out Cross-ValidationComparing the K-Fold and LOO MethodsCross-Validation for Deep Learning ModelsKeras Wrapper with scikit-learnExercise 4.01: Building the Keras Wrapper with scikit-learn for a Regression ProblemCross-Validation with scikit-learnCross-Validation Iterators in scikit-learnExercise 4.02: Evaluating Deep Neural Networks with Cross-ValidationActivity 4.01: Model Evaluation Using Cross-Validation for an Advanced Fibrosis Diagnosis ClassifierModel Selection with Cross-ValidationCross-Validation for Model Evaluation versus Model SelectionExercise 4.03: Writing User-Defined Functions to Implement Deep Learning Models with Cross-ValidationActivity 4.02: Model Selection Using Cross-Validation for the Advanced Fibrosis Diagnosis ClassifierActivity 4.03: Model Selection Using Cross-validation on a Traffic Volume DatasetSummary
IntroductionRegularizationThe Need for RegularizationReducing Overfitting with RegularizationL1 and L2 RegularizationL1 and L2 Regularization FormulationL1 and L2 Regularization Implementation in KerasActivity 5.01: Weight Regularization on an Avila Pattern ClassifierDropout RegularizationPrinciples of Dropout RegularizationReducing Overfitting with DropoutExercise 5.01: Dropout Implementation in KerasActivity 5.02: Dropout Regularization on the Traffic Volume DatasetOther Regularization MethodsEarly StoppingExercise 5.02: Implementing Early Stopping in KerasData AugmentationAdding NoiseHyperparameter Tuning with scikit-learnGrid Search with scikit-learnRandomized Search with scikit-learnActivity 5.03: Hyperparameter Tuning on the Avila Pattern ClassifierSummary
IntroductionAccuracyExercise 6.01: Calculating Null Accuracy on a Pacific Hurricanes DatasetAdvantages and Limitations of AccuracyImbalanced DatasetsWorking with Imbalanced DatasetsConfusion MatrixMetrics Computed from a Confusion MatrixExercise 6.02: Computing Accuracy and Null Accuracy with APS Failure for Scania Trucks DataActivity 6.01: Computing the Accuracy and Null Accuracy of a Neural Network When We Change the Train/Test SplitExercise 6.03: Deriving and Computing Metrics Based on a Confusion MatrixActivity 6.02: Calculating the ROC Curve and AUC ScoreSummary
IntroductionComputer VisionConvolutional Neural NetworksThe Architecture of a CNNInput ImageConvolution LayerThe Pooling LayerFlatteningImage AugmentationAdvantages of Image AugmentationExercise 7.01: Building a CNN and Identifying Images of Cars and FlowersActivity 7.01: Amending Our Model with Multiple Layers and the Use of softmaxExercise 7.02: Amending Our Model by Reverting to the Sigmoid Activation FunctionExercise 7.03: Changing the Optimizer from Adam to SGDExercise 7.04: Classifying a New ImageActivity 7.02: Classifying a New ImageSummary
IntroductionPre-Trained Sets and Transfer LearningFeature ExtractionFine-Tuning a Pre-Trained Network The ImageNet DatasetSome Pre-Trained Networks in KerasExercise 8.01: Identifying an Image Using the VGG16 NetworkActivity 8.01: Using the VGG16 Network to Train a Deep Learning Network to Identify ImagesExercise 8.02: Classifying Images That Are Not Present in the ImageNet DatabaseExercise 8.03: Fine-Tuning the VGG16 ModelExercise 8.04: Image Classification with ResNetActivity 8.02: Image Classification with ResNet Summary

IntroductionSequential Memory and Sequential Modeling Recurrent Neural Networks (RNNs)The Vanishing Gradient ProblemA Brief Explanation of the Exploding Gradient ProblemLong Short-Term Memory (LSTM)Exercise 9.01: Predicting the Trend of Alphabet's Stock Price Using an LSTM with 50 Units (Neurons)Activity 9.01: Predicting the Trend of Amazon's Stock Price Using an LSTM with 50 Units (Neurons)Exercise 9.02: Predicting the Trend of Alphabet's Stock Price Using an LSTM with 100 unitsActivity 9.02: Predicting Amazon's Stock Price with Added RegularizationActivity 9.03: Predicting the Trend of Amazon's Stock Price Using an LSTM with an Increasing Number of LSTM Neurons (100 Units)Summary
1. Introduction to Machine Learning with KerasActivity 1.01: Adding Regularization to the Model2. Machine Learning versus Deep LearningActivity 2.01: Creating a Logistic Regression Model Using Keras3. Deep Learning with KerasActivity 3.01: Building a Single-Layer Neural Network for Performing Binary ClassificationActivity 3.02: Advanced Fibrosis Diagnosis with Neural Networks4. Evaluating Your Model with Cross-Validation Using Keras WrappersActivity 4.01: Model Evaluation Using Cross-Validation for an Advanced Fibrosis Diagnosis ClassifierActivity 4.02: Model Selection Using Cross-Validation for the Advanced Fibrosis Diagnosis ClassifierActivity 4.03: Model Selection Using Cross-validation on a Traffic Volume Dataset5. Improving Model AccuracyActivity 5.01: Weight Regularization on an Avila Pattern ClassifierActivity 5.02: Dropout Regularization on the Traffic Volume DatasetActivity 5.03: Hyperparameter Tuning on the Avila Pattern Classifier6. Model EvaluationActivity 6.01: Computing the Accuracy and Null Accuracy of a Neural Network When We Change the Train/Test SplitActivity 6.02: Calculating the ROC Curve and AUC Score7. Computer Vision with Convolutional Neural NetworksActivity 7.01: Amending Our Model with Multiple Layers and the Use of softmaxActivity 7.02: Classifying a New Image8. Transfer Learning and Pre-Trained ModelsActivity 8.01: Using the VGG16 Network to Train a Deep Learning Network to Identify ImagesActivity 8.02: Image Classification with ResNet 9. Sequential Modeling with Recurrent Neural NetworksActivity 9.01: Predicting the Trend of Amazon's Stock Price Using an LSTM with 50 Units (Neurons)Activity 9.02: Predicting Amazon's Stock Price with Added RegularizationActivity 9.03: Predicting the Trend of Amazon's Stock Price Using an LSTM with an Increasing Number of LSTM Neurons (100 Units)

Content preview from The Deep Learning with Keras Workshop

Appendix

1. Introduction to Machine Learning with Keras

Activity 1.01: Adding Regularization to the Model

In this activity, we will utilize the same logistic regression model from the scikit-learn package. This time, however, we will add regularization to the model and search for the optimum regularization parameter - a process often called hyperparameter tuning. After training the models, we will test the predictions and compare the model evaluation metrics to the ones that were produced by the baseline model and the model without regularization.

Load the feature data from Exercise 1.03, Appropriate Representation of the Data, and the target data from Exercise 1.02, Cleaning the Data:
import pandas as pd
feats = pd.read_csv('../data/OSI_feats_e3.csv') ...