book

Deep Learning with R, Second Edition

Name: Deep Learning with R, Second Edition
Author: Tomasz Kalinowski
ISBN: 9781633439849

by Tomasz Kalinowski

October 2022

Intermediate to advanced

568 pages

16h 35m

English

Manning Publications

Audiobook available

Read now

Unlock full access

Cover
Title Page
Copyright
Preface
Acknowledgments
About This Book
About the Authors
Chapter 1: What Is Deep Learning?
1.1 Artificial Intelligence, Machine Learning, and Deep LearningArtificial IntelligenceMachine LearningLearning Rules and Representations from DataThe “Deep” in “Deep Learning”Understanding How Deep Learning Works, in Three FiguresWhat Deep Learning Has Achieved So FarDon’t Believe the Short-Term HypeThe Promise of AI1.2 Before Deep Learning: A Brief History of Machine LearningProbabilistic ModelingEarly Neural NetworksKernel MethodsDecision Trees, Random Forests, and Gradient-Boosting MachinesBack to Neural NetworksWhat Makes Deep Learning Different?The Modern Machine Learning Landscape1.3 Why Deep Learning? Why Now?HardwareDataAlgorithmsA New Wave of InvestmentThe Democratization of Deep LearningWill It Last?
Chapter 2: The Mathematical Building Blocks of Neural Networks
2.1 A First Look at a Neural Network2.2 Data Representations for Neural NetworksScalars (Rank 0 Tensors)Vectors (Rank 1 Tensors)Matrices (Rank 2 Tensors)Rank 3 and Higher-Rank TensorsKey AttributesManipulating Tensors in RThe Notion of Data BatchesReal-World Examples of Data TensorsVector DataTime-Series Data or Sequence DataImage DataVideo Data2.3 The Gears of Neural Networks: Tensor OperationsElement-Wise OperationsBroadcastingTensor ProductTensor ReshapingGeometric Interpretation of Tensor OperationsA Geometric Interpretation of Deep Learning2.4 The Engine of Neural Networks: Gradient-Based OptimizationWhat’s a Derivative?Derivative of a Tensor Operation: The gradientStochastic Gradient DescentChaining Derivatives: The Backpropagation Algorithm2.5 Looking Back at Our First ExampleReimplementing our First Example from Scratch in TensorFlowRunning One Training StepThe Full Training LoopEvaluating the Model
Chapter 3: Introduction to Keras and Tensorflow
3.1 What’s Tensorflow?3.2 What’s Keras?3.3 Keras and Tensorflow: A Brief History3.4 Python and R Interfaces: A Brief History3.5 Setting Up A Deep Learning WorkspaceInstalling Keras and TensorFlow3.6 First Steps with TensorflowTensorflow Tensors3.7 Tensor AttributesTensor Shape and ReshapingTensor SlicingTensor BroadcastingThe tf ModuleConstant Tensors and VariablesTensor Operations: Doing Math in TensorFlowA Second Look at the GradientTape APIAn End-to-End Example: A Linear Classifier in Pure TensorFlow3.8 Anatomy of a Neural Network: Understanding Core Keras ApisLayers: The Building Blocks of Deep LearningFrom Layers to ModelsThe “Compile” Step: Configuring the Learning ProcessPicking a Loss FunctionUnderstanding the fit() MethodMonitoring Loss and Metrics on Validation DataInference: Using a Model After Training

Chapter 4: Getting Started with Neural Networks: Classification and Regression
4.1 Classifying Movie Reviews: A Binary Classification ExampleThe IMDB DatasetPreparing the DataBuilding Your ModelValidating Your ApproachUsing a Trained Model to Generate Predictions on New DataFurther ExperimentsWrapping Up4.2 Classifying Newswires: A Multiclass Classification ExampleThe Reuters DatasetPreparing the DataBuilding Your ModelValidating Your ApproachGenerating Predictions on New DataA Different Way to Handle the Labels and the LossThe Importance of Having Sufficiently Large Intermediate LayersFurther ExperimentsWrapping Up4.3 Predicting House Prices: A Regression ExampleThe Boston Housing Price DatasetPreparing the Data 123 Building Your ModelValidating Your Approach Using K-fold ValidationGenerating Predictions on New DataWrapping Up
Chapter 5: Fundamentals of Machine Learning
5.1 Generalization: The Goal of Machine LearningUnderfitting and OverfittingThe Nature of Generalization in Deep Learning5.2 Evaluating Machine Learning ModelsTraining, Validation, and Test SetsBeating a Common-Sense BaselineThings to Keep in Mind About Model Evaluation5.3 Improving Model FitTuning Key Gradient Descent ParametersLeveraging Better Architecture PriorsIncreasing Model Capacity5.4 Improving GeneralizationDataset CurationFeature EngineeringUsing Early StoppingRegularizing Your Model
Chapter 6: The Universal Workflow of Machine Learning
6.1 Define The TaskFrame the ProblemCollect a DatasetUnderstand Your DataChoose a Measure of Success6.2 Develop a ModelPrepare the DataChoose an Evaluation ProtocolBeat a BaselineScale Up: Develop a Model that OverfitsRegularize and Tune Your Model6.3 Deploy the ModelExplain Your Work to Stakeholders and Set ExpectationsShip an Inference ModelMonitor Your Model in the WildMaintain Your Model
Chapter 7: Working with Keras: A Deep Dive
7.1 A Spectrum of Workflows7.2 Different Ways to Build Keras ModelsThe Sequential ModelThe Functional APISubclassing the Model ClassMixing and Matching Different ComponentsRemember: Use the Right Tool for the Job7.3 Using Built-In Training and Evaluation LoopsWriting Your Own MetricsUsing CallbacksWriting Your Own CallbacksMonitoring and Visualization with Tensorboard7.4 Writing Your Own Training and Evaluation LoopsTraining vs. InferenceLow-Level Usage of MetricsA Complete Training and Evaluation LoopMake It Fast with tf_function()Leveraging fit() with a Custom Training Loop
Chapter 8: Introduction to Deep Learning for Computer Vision
8.1 Introduction to ConvnetsThe Convolution OperationThe Max-Pooling Operation8.2 Training a Convnet from Scratch on a Small DatasetThe Relevance of Deep Learning for Small Data ProblemsDownloading the DataBuilding the ModelData PreprocessingUsing Data Augmentation8.3 Leveraging a Pretrained ModelFeature Extraction with a Pretrained ModelFine-Tuning a Pretrained Model
Chapter 9: Advanced Deep Learning for Computer Vision
9.1 Three Essential Computer Vision Tasks9.2 An Image Segmentation Example9.3 Modern Convnet Architecture PatternsModularity, Hierarchy, and ReuseResidual ConnectionsBatch NormalizationDepthwise Separable ConvolutionsPutting It Together: A Mini Xception-Like Model9.4 Interpreting What Convnets LearnVisualizing Intermediate ActivationsVisualizing Convnet FiltersVisualizing Heatmaps of Class Activation
Chapter 10: Deep Learning for Time Series
10.1 Different Kinds of Time-Series Tasks10.2 A Temperature-Forecasting ExamplePreparing the DataA Common-Sense, Non–Machine Learning BaselineLet’s Try a Basic Machine Learning ModelLet’s Try a 1D Convolutional ModelA First Recurrent Baseline10.3 Understanding Recurrent Neural NetworksA Recurrent Layer in Keras10.4 Advanced Use of Recurrent Neural NetworksUsing Recurrent Dropout to Fight OverfittingStacking Recurrent LayersUsing Bidirectional RNNsGoing Even Further
Chapter 11: Deep Learning for Text
11.1 Natural Language Processing: The Bird’s-Eye View11.2 Preparing Text DataText StandardizationText Splitting (Tokenization)Vocabulary IndexingUsing layer_text_vectorization11.3 Two Approaches for Representing Groups of Words: Sets and SequencesPreparing the IMDB Movie Reviews DataProcessing Words as a Set: The Bag-of-Words ApproachProcessing Words as a Sequence: The Sequence Model Approach11.4 The Transformer ArchitectureUnderstanding Self-AttentionMulti-Head AttentionThe Transformer EncoderWhen to Use Sequence Models Over Bag-of-Words Models11.5 Beyond Text Classification: Sequence-to-Sequence LearningA machine Translation ExampleSequence-to-Sequence Learning with RNNsSequence-to-Sequence Learning with Transformer
Chapter 12: Generative Deep Learning
12.1 Text GenerationA Brief History of Generative Deep Learning for Sequence GenerationHow do you Generate Sequence Data?The Importance of the Sampling StrategyImplementing Text Generation with KerasA Text-Generation Callback with Variable-Temperature SamplingWrapping Up12.2 DeepdreamImplementing Deepdream in KerasWrapping Up12.3 Neural Style TransferThe Content LossThe Style LossNeural Style Transfer in KerasWrapping Up12.4 Generating Images with Variational AutoencodersSampling from Latent Spaces of ImagesConcept Vectors for Image EditingVariational AutoencodersImplementing a VAE with KerasWrapping Up12.5 Introduction to Generative Adversarial NetworksA Schematic GAN ImplementationA Bag of TricksGetting Our Hands on the Celeba DatasetThe DiscriminatorThe GeneratorThe Adversarial NetworkWrapping Up
Chapter 13: Best Practices for the Real World
13.1 Getting the Most Out of Your ModelsHyperparameter OptimizationModel Ensembling13.2 Scaling-Up Model TrainingSpeeding Up Training on GPU with Mixed PrecisionMulti-GPU TrainingTPU Training
Chapter 14: Conclusions
14.1 Key Concepts in ReviewVarious Approaches to AIWhat Makes Deep Learning Special within the Field of Machine LearningHow to Think About Deep LearningKey Enabling TechnologiesThe Universal Machine Learning WorkflowKey Network ArchitecturesThe Space of Possibilities14.2 The Limitations of Deep LearningThe Risk of Anthropomorphizing Machine Learning ModelsAutomatons vs. Intelligent AgentsLocal Generalization vs. Extreme GeneralizationThe Purpose of IntelligenceClimbing the Spectrum of Generalization14.3 Setting The Course Toward Greater Generality in AIOn the Importance of Setting the Right Objective: The Shortcut RuleA New Target14.4 Implementing Intelligence: The Missing IngredientsIntelligence as Sensitivity to Abstract AnalogiesThe Two Poles of AbstractionThe Two Poles of AbstractionThe Missing Half of the Picture14.5 The Future of Deep LearningModels as ProgramsMachine Learning vs. Program SynthesisBlending Together Deep Learning and Program SynthesisLifelong Learning and Modular Subroutine ReuseThe Long-Term Vision14.6 Staying Up-to-Date in a Fast-Moving FieldPractice on Real-World Problems Using KaggleRead About the Latest Developments on arXivExplore the Keras Ecosystem14.7 Final Words
Appendix: Python Primer for R Users
Index

Content preview from Deep Learning with R, Second Edition

2 The mathematical building blocks of neural networks

This chapter covers

A first example of a neural network
Tensors and tensor operations
How neural networks learn via backpropagation and gradient descent

Understanding deep learning requires familiarity with many simple mathematical concepts: tensors, tensor operations, differentiation, gradient descent, and so on. Our goal in this chapter will be to build up your intuition about these notions without getting overly technical. In particular, we’ll steer away from mathematical notation, which can introduce unnecessary barriers for those without any mathematics background and isn’t necessary to explain things well. The most precise, unambiguous description of a mathematical operation is ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633439849Publisher Support Publisher Website Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Deep Learning with R, Second Edition

by Tomasz Kalinowski

2 The mathematical building blocks of neural networks

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.