book

Deep Learning with R, Second Edition

by Sigrid Keydana, Francois Chollet, Tomasz Kalinowski, J.J. Allaire

October 2022

Intermediate to advanced

568 pages

16h 35m

English

Manning Publications

Read now

Unlock full access

1.1 Artificial Intelligence, Machine Learning, and Deep LearningArtificial IntelligenceMachine LearningLearning Rules and Representations from DataThe “Deep” in “Deep Learning”Understanding How Deep Learning Works, in Three FiguresWhat Deep Learning Has Achieved So FarDon’t Believe the Short-Term HypeThe Promise of AI1.2 Before Deep Learning: A Brief History of Machine LearningProbabilistic ModelingEarly Neural NetworksKernel MethodsDecision Trees, Random Forests, and Gradient-Boosting MachinesBack to Neural NetworksWhat Makes Deep Learning Different?The Modern Machine Learning Landscape1.3 Why Deep Learning? Why Now?HardwareDataAlgorithmsA New Wave of InvestmentThe Democratization of Deep LearningWill It Last?
2.1 A First Look at a Neural Network2.2 Data Representations for Neural NetworksScalars (Rank 0 Tensors)Vectors (Rank 1 Tensors)Matrices (Rank 2 Tensors)Rank 3 and Higher-Rank TensorsKey AttributesManipulating Tensors in RThe Notion of Data BatchesReal-World Examples of Data TensorsVector DataTime-Series Data or Sequence DataImage DataVideo Data2.3 The Gears of Neural Networks: Tensor OperationsElement-Wise OperationsBroadcastingTensor ProductTensor ReshapingGeometric Interpretation of Tensor OperationsA Geometric Interpretation of Deep Learning2.4 The Engine of Neural Networks: Gradient-Based OptimizationWhat’s a Derivative?Derivative of a Tensor Operation: The gradientStochastic Gradient DescentChaining Derivatives: The Backpropagation Algorithm2.5 Looking Back at Our First ExampleReimplementing our First Example from Scratch in TensorFlowRunning One Training StepThe Full Training LoopEvaluating the Model
3.1 What’s Tensorflow?3.2 What’s Keras?3.3 Keras and Tensorflow: A Brief History3.4 Python and R Interfaces: A Brief History3.5 Setting Up A Deep Learning WorkspaceInstalling Keras and TensorFlow3.6 First Steps with TensorflowTensorflow Tensors3.7 Tensor AttributesTensor Shape and ReshapingTensor SlicingTensor BroadcastingThe tf ModuleConstant Tensors and VariablesTensor Operations: Doing Math in TensorFlowA Second Look at the GradientTape APIAn End-to-End Example: A Linear Classifier in Pure TensorFlow3.8 Anatomy of a Neural Network: Understanding Core Keras ApisLayers: The Building Blocks of Deep LearningFrom Layers to ModelsThe “Compile” Step: Configuring the Learning ProcessPicking a Loss FunctionUnderstanding the fit() MethodMonitoring Loss and Metrics on Validation DataInference: Using a Model After Training

4.1 Classifying Movie Reviews: A Binary Classification ExampleThe IMDB DatasetPreparing the DataBuilding Your ModelValidating Your ApproachUsing a Trained Model to Generate Predictions on New DataFurther ExperimentsWrapping Up4.2 Classifying Newswires: A Multiclass Classification ExampleThe Reuters DatasetPreparing the DataBuilding Your ModelValidating Your ApproachGenerating Predictions on New DataA Different Way to Handle the Labels and the LossThe Importance of Having Sufficiently Large Intermediate LayersFurther ExperimentsWrapping Up4.3 Predicting House Prices: A Regression ExampleThe Boston Housing Price DatasetPreparing the Data 123 Building Your ModelValidating Your Approach Using K-fold ValidationGenerating Predictions on New DataWrapping Up
5.1 Generalization: The Goal of Machine LearningUnderfitting and OverfittingThe Nature of Generalization in Deep Learning5.2 Evaluating Machine Learning ModelsTraining, Validation, and Test SetsBeating a Common-Sense BaselineThings to Keep in Mind About Model Evaluation5.3 Improving Model FitTuning Key Gradient Descent ParametersLeveraging Better Architecture PriorsIncreasing Model Capacity5.4 Improving GeneralizationDataset CurationFeature EngineeringUsing Early StoppingRegularizing Your Model
6.1 Define The TaskFrame the ProblemCollect a DatasetUnderstand Your DataChoose a Measure of Success6.2 Develop a ModelPrepare the DataChoose an Evaluation ProtocolBeat a BaselineScale Up: Develop a Model that OverfitsRegularize and Tune Your Model6.3 Deploy the ModelExplain Your Work to Stakeholders and Set ExpectationsShip an Inference ModelMonitor Your Model in the WildMaintain Your Model
7.1 A Spectrum of Workflows7.2 Different Ways to Build Keras ModelsThe Sequential ModelThe Functional APISubclassing the Model ClassMixing and Matching Different ComponentsRemember: Use the Right Tool for the Job7.3 Using Built-In Training and Evaluation LoopsWriting Your Own MetricsUsing CallbacksWriting Your Own CallbacksMonitoring and Visualization with Tensorboard7.4 Writing Your Own Training and Evaluation LoopsTraining vs. InferenceLow-Level Usage of MetricsA Complete Training and Evaluation LoopMake It Fast with tf_function()Leveraging fit() with a Custom Training Loop
8.1 Introduction to ConvnetsThe Convolution OperationThe Max-Pooling Operation8.2 Training a Convnet from Scratch on a Small DatasetThe Relevance of Deep Learning for Small Data ProblemsDownloading the DataBuilding the ModelData PreprocessingUsing Data Augmentation8.3 Leveraging a Pretrained ModelFeature Extraction with a Pretrained ModelFine-Tuning a Pretrained Model
9.1 Three Essential Computer Vision Tasks9.2 An Image Segmentation Example9.3 Modern Convnet Architecture PatternsModularity, Hierarchy, and ReuseResidual ConnectionsBatch NormalizationDepthwise Separable ConvolutionsPutting It Together: A Mini Xception-Like Model9.4 Interpreting What Convnets LearnVisualizing Intermediate ActivationsVisualizing Convnet FiltersVisualizing Heatmaps of Class Activation
10.1 Different Kinds of Time-Series Tasks10.2 A Temperature-Forecasting ExamplePreparing the DataA Common-Sense, Non–Machine Learning BaselineLet’s Try a Basic Machine Learning ModelLet’s Try a 1D Convolutional ModelA First Recurrent Baseline10.3 Understanding Recurrent Neural NetworksA Recurrent Layer in Keras10.4 Advanced Use of Recurrent Neural NetworksUsing Recurrent Dropout to Fight OverfittingStacking Recurrent LayersUsing Bidirectional RNNsGoing Even Further
11.1 Natural Language Processing: The Bird’s-Eye View11.2 Preparing Text DataText StandardizationText Splitting (Tokenization)Vocabulary IndexingUsing layer_text_vectorization11.3 Two Approaches for Representing Groups of Words: Sets and SequencesPreparing the IMDB Movie Reviews DataProcessing Words as a Set: The Bag-of-Words ApproachProcessing Words as a Sequence: The Sequence Model Approach11.4 The Transformer ArchitectureUnderstanding Self-AttentionMulti-Head AttentionThe Transformer EncoderWhen to Use Sequence Models Over Bag-of-Words Models11.5 Beyond Text Classification: Sequence-to-Sequence LearningA machine Translation ExampleSequence-to-Sequence Learning with RNNsSequence-to-Sequence Learning with Transformer
12.1 Text GenerationA Brief History of Generative Deep Learning for Sequence GenerationHow do you Generate Sequence Data?The Importance of the Sampling StrategyImplementing Text Generation with KerasA Text-Generation Callback with Variable-Temperature SamplingWrapping Up12.2 DeepdreamImplementing Deepdream in KerasWrapping Up12.3 Neural Style TransferThe Content LossThe Style LossNeural Style Transfer in KerasWrapping Up12.4 Generating Images with Variational AutoencodersSampling from Latent Spaces of ImagesConcept Vectors for Image EditingVariational AutoencodersImplementing a VAE with KerasWrapping Up12.5 Introduction to Generative Adversarial NetworksA Schematic GAN ImplementationA Bag of TricksGetting Our Hands on the Celeba DatasetThe DiscriminatorThe GeneratorThe Adversarial NetworkWrapping Up
13.1 Getting the Most Out of Your ModelsHyperparameter OptimizationModel Ensembling13.2 Scaling-Up Model TrainingSpeeding Up Training on GPU with Mixed PrecisionMulti-GPU TrainingTPU Training
14.1 Key Concepts in ReviewVarious Approaches to AIWhat Makes Deep Learning Special within the Field of Machine LearningHow to Think About Deep LearningKey Enabling TechnologiesThe Universal Machine Learning WorkflowKey Network ArchitecturesThe Space of Possibilities14.2 The Limitations of Deep LearningThe Risk of Anthropomorphizing Machine Learning ModelsAutomatons vs. Intelligent AgentsLocal Generalization vs. Extreme GeneralizationThe Purpose of IntelligenceClimbing the Spectrum of Generalization14.3 Setting The Course Toward Greater Generality in AIOn the Importance of Setting the Right Objective: The Shortcut RuleA New Target14.4 Implementing Intelligence: The Missing IngredientsIntelligence as Sensitivity to Abstract AnalogiesThe Two Poles of AbstractionThe Two Poles of AbstractionThe Missing Half of the Picture14.5 The Future of Deep LearningModels as ProgramsMachine Learning vs. Program SynthesisBlending Together Deep Learning and Program SynthesisLifelong Learning and Modular Subroutine ReuseThe Long-Term Vision14.6 Staying Up-to-Date in a Fast-Moving FieldPractice on Real-World Problems Using KaggleRead About the Latest Developments on arXivExplore the Keras Ecosystem14.7 Final Words

Content preview from Deep Learning with R, Second Edition

11 Deep learning for text

This chapter covers

Preprocessing text data for machine learning applications
Bag-of-words approaches and sequence-modeling approaches for text processing
The Transformer architecture
Sequence-to-sequence learning

11.1 Natural language processing: The bird’s-eye view

In computer science, we refer to human languages, like English or Mandarin, as “natural” languages, to distinguish them from languages that were designed for machines, like Assembly, LISP, or XML. Every machine language was designed: its starting point was a human engineer writing down a set of formal rules to describe what statements you could make in that language and what they meant. Rules came first, and people started using the language only once ...