book

Modern Computer Vision with PyTorch

Name: Modern Computer Vision with PyTorch
ISBN: 9781839213472

by V Kishore Ayyadevara, Yeshwanth Reddy

November 2020

Intermediate to advanced

824 pages

17h 9m

English

Packt Publishing

Read now

Unlock full access

Title Page
Copyright and Credits
Modern Computer Vision with PyTorch
Dedication
About Packt
Why subscribe?
Contributors
About the authorsAbout the reviewerPackt is searching for authors like you
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchReviews
Section 1 - Fundamentals of Deep Learning for Computer Vision
Artificial Neural Network Fundamentals
Comparing AI and traditional machine learningLearning about the artificial neural network building blocksImplementing feedforward propagationCalculating the hidden layer unit valuesApplying the activation functionCalculating the output layer valuesCalculating loss valuesCalculating loss during continuous variable predictionCalculating loss during categorical variable predictionFeedforward propagation in codeActivation functions in codeLoss functions in codeImplementing backpropagationGradient descent in codeImplementing backpropagation using the chain rulePutting feedforward propagation and backpropagation togetherUnderstanding the impact of the learning rate Summarizing the training process of a neural networkSummaryQuestions
PyTorch Fundamentals
Installing PyTorchPyTorch tensorsInitializing a tensorOperations on tensorsAuto gradients of tensor objectsAdvantages of PyTorch's tensors over NumPy's ndarraysBuilding a neural network using PyTorchDataset, DataLoader, and batch sizePredicting on new data pointsImplementing a custom loss functionFetching the values of intermediate layersUsing a sequential method to build a neural networkSaving and loading a PyTorch modelstate dictSavingLoadingSummaryQuestions
Building a Deep Neural Network with PyTorch
Representing an imageConverting images into structured arrays and scalarsWhy leverage neural networks for image analysis?Preparing our data for image classificationTraining a neural networkScaling a dataset to improve model accuracyUnderstanding the impact of varying the batch sizeBatch size of 32Batch size of 10,000Understanding the impact of varying the loss optimizerUnderstanding the impact of varying the learning rateImpact of the learning rate on a scaled datasetHigh learning rateMedium learning rateLow learning rateParameter distribution across layers for different learning ratesImpact of varying the learning rate on a non-scaled datasetUnderstanding the impact of learning rate annealingBuilding a deeper neural networkUnderstanding the impact of batch normalizationVery small input values without batch normalizationVery small input values with batch normalizationThe concept of overfittingImpact of adding dropoutImpact of regularizationL1 regularizationL2 regularizationSummaryQuestions

Section 2 - Object Classification and Detection
Introducing Convolutional Neural Networks
The problem with traditional deep neural networksBuilding blocks of a CNNConvolutionFilterStrides and padding StridesPaddingPoolingPutting them all togetherHow convolution and pooling help in image translationImplementing a CNN Building a CNN-based architecture using PyTorchForward propagating the output in PythonClassifying images using deep CNNsImplementing data augmentationImage augmentationsAffine transformationsChanging the brightnessAdding noisePerforming a sequence of augmentationsPerforming data augmentation on a batch of images and the need for collate_fnData augmentation for image translationVisualizing the outcome of feature learningBuilding a CNN for classifying real-world imagesImpact on the number of images used for trainingSummaryQuestions
Transfer Learning for Image Classification
Introducing transfer learningUnderstanding VGG16 architectureUnderstanding ResNet architectureImplementing facial key point detection2D and 3D facial key point detectionMulti-task learning – Implementing age estimation and gender classificationIntroducing the torch_snippets librarySummaryQuestions
Practical Aspects of Image Classification
Generating CAMsUnderstanding the impact of data augmentation and batch normalizationCoding up road sign detectionPractical aspects to take care of during model implementationDealing with imbalanced dataThe size of the object within an imageDealing with the difference between training and validation dataThe number of nodes in the flatten layerImage sizeLeveraging OpenCV utilitiesSummaryQuestions
Basics of Object Detection
Introducing object detectionCreating a bounding box ground truth for trainingInstalling the image annotation toolUnderstanding region proposalsLeveraging SelectiveSearch to generate region proposalsImplementing SelectiveSearch to generate region proposalsUnderstanding IoUNon-max suppressionMean average precisionTraining R-CNN-based custom object detectorsWorking details of R-CNNImplementing R-CNN for object detection on a custom datasetDownloading the datasetPreparing the datasetFetching region proposals and the ground truth of offsetCreating the training dataR-CNN network architecturePredict on a new imageTraining Fast R-CNN-based custom object detectorsWorking details of Fast R-CNNImplementing Fast R-CNN for object detection on a custom datasetSummaryQuestions
Advanced Object Detection
Components of modern object detection algorithmsAnchor boxesRegion Proposal NetworkClassification and regressionTraining Faster R-CNN on a custom datasetWorking details of YOLOTraining YOLO on a custom datasetInstalling DarknetSetting up the dataset formatConfiguring the architectureTraining and testing the modelWorking details of SSDComponents in SSD codeSSD300MultiBoxLossTraining SSD on a custom datasetSummaryTest your understanding
Image Segmentation
Exploring the U-Net architecturePerforming upscalingImplementing semantic segmentation using U-NetExploring the Mask R-CNN architectureRoI AlignMask headImplementing instance segmentation using Mask R-CNNPredicting multiple instances of multiple classesSummaryQuestions
Applications of Object Detection and Segmentation
Multi-object instance segmentationFetching and preparing dataTraining the model for instance segmentationMaking inferences on a new imageHuman pose detectionCrowd countingCoding up crowd countingImage colorization3D object detection with point cloudsTheoryInput encodingOutput encodingTraining the YOLO model for 3D object detectionData formatData inspectionTrainingTestingSummary
Section 3 - Image Manipulation
Autoencoders and Image Manipulation
Understanding autoencodersImplementing vanilla autoencodersUnderstanding convolutional autoencodersGrouping similar images using t-SNEUnderstanding variational autoencodersWorking of VAEKL divergenceBuilding a VAEPerforming an adversarial attack on imagesPerforming neural style transferGenerating deep fakesSummaryQuestions
Image Generation Using GANs
Introducing GANsUsing GANs to generate handwritten digitsUsing DCGANs to generate face imagesImplementing conditional GANsSummaryQuestions
Advanced GANs to Manipulate Images
Leveraging the Pix2Pix GANLeveraging CycleGANLeveraging StyleGAN on custom imagesSuper-resolution GANArchitectureCoding SRGANSummaryQuestions
Section 4 - Combining Computer Vision with Other Techniques
Training with Minimal Data Points
Implementing zero-shot learningCoding zero-shot learningImplementing few-shot learningBuilding a Siamese networkCoding Siamese networksWorking details of prototypical networksWorking details of relation networksSummaryQuestions
Combining Computer Vision and NLP Techniques
Introducing RNNsThe idea behind the need for RNN architectureExploring the structure of an RNNWhy store memory?Introducing LSTM architectureThe working details of LSTMImplementing LSTM in PyTorchImplementing image captioningImage captioning in codeTranscribing handwritten imagesThe working details of CTC lossCalculating the CTC loss valueHandwriting transcription in codeObject detection using DETRThe working details of transformersBasics of transformersThe working details of DETRDetection with transformers in codeSummaryQuestions
Combining Computer Vision and Reinforcement Learning
Learning the basics of reinforcement learningCalculating the state valueCalculating the state-action valueImplementing Q-learningQ-valueUnderstanding the Gym environmentBuilding a Q-tableLeveraging exploration-exploitationImplementing deep Q-learningImplementing deep Q-learning with the fixed targets modelCoding up an agent to play PongImplementing an agent to perform autonomous drivingInstalling the CARLA environmentInstall the CARLA binariesInstalling the CARLA Gym environmentTraining a self-driving agentmodel.pyactor.pyTraining DQN with fixed targetsSummaryQuestions
Moving a Model to Production
Understanding the basics of an APICreating an API and making predictions on a local serverInstalling the API module and dependenciesServing an image classifierfmnist.pyserver.pyRunning the serverMoving the API to the cloudComparing Docker containers and Docker imagesCreating a Docker containerCreating the requirements.txt fileCreating a DockerfileBuilding a Docker image and creating a Docker containerShipping and running the Docker container in the cloudConfiguring AWSCreating a Docker repository on AWS ECR and pushing the imageCreating an EC2 instancePulling the image and building the Docker containerSummary
Using OpenCV Utilities for Image Analysis
Drawing bounding boxes around words in an imageDetecting lanes in an image of a roadDetecting objects based on colorBuilding a panoramic view of imagesDetecting the number plate of a carSummary
Appendix
Chapter 1 - Artificial Neural Network FundamentalsChapter 2 - PyTorch FundamentalsChapter 3 - Building a Deep Neural Network with PyTorchChapter 4 - Introducing Convolutional Neural NetworksChapter 5 - Transfer Learning for Image ClassificationChapter 6 - Practical Aspects of Image ClassificationChapter 7 - Basics of Object DetectionChapter 8 - Advanced Object DetectionChapter 9 - Image SegmentationChapter 11 - Autoencoders and Image ManipulationChapter 12 - Image Generation Using GANsChapter 13 - Advanced GANs to Manipulate ImagesChapter 14 - Training with Minimal Data PointsChapter 15 - Combining Computer Vision and NLP TechniquesChapter 16 - Combining Computer Vision and Reinforcement Learning
Other Books You May Enjoy
Leave a review - let other readers know what you think

Content preview from Modern Computer Vision with PyTorch

Combining Computer Vision and NLP Techniques

In the previous chapter, we learned about leveraging novel architectures when there are a minimal number of data points. In this chapter, we will switch gears and learn about how a Convolutional Neural Network (CNN) can be used in conjunction with algorithms in the broad family of Recurrent Neural Networks (RNNs), which are heavily used (as of the time of writing this book) in Natural Language Processing (NLP) to develop solutions that leverage both computer vision and NLP.

To understand combining CNNs and RNNs, we will first learn about how RNNs work and their variants – primarily Long Short-Term Memory (LSTM) – to understand how they are applied to predict annotations given an image as input. ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Modern Computer Vision with PyTorch - Second Edition

Publisher Resources

ISBN: 9781839213472

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Modern Computer Vision with PyTorch

by V Kishore Ayyadevara, Yeshwanth Reddy

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.