book

Modern Computer Vision with PyTorch - Second Edition

Name: Modern Computer Vision with PyTorch - Second Edition
ISBN: 9781803231334

by V Kishore Ayyadevara, Yeshwanth Reddy

June 2024

Intermediate to advanced

746 pages

17h 38m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book coversTo get the most out of this bookGet in touchFree Benefits with Your BookHow to Unlock
Section 1: Fundamentals of Deep Learning for Computer Vision
Artificial Neural Network Fundamentals
Free Benefits with Your BookComparing AI and traditional machine learningLearning about the ANN building blocksImplementing feedforward propagationCalculating the hidden layer unit valuesApplying the activation functionCalculating the output layer valuesCalculating loss valuesCalculating loss during continuous variable predictionCalculating loss during categorical variable predictionFeedforward propagation in codeActivation functions in codeLoss functions in codeImplementing backpropagationGradient descent in codeImplementing backpropagation using the chain rulePutting feedforward propagation and backpropagation togetherUnderstanding the impact of the learning rateLearning rate of 0.01Learning rate of 0.1Learning rate of 1Summarizing the training process of a neural networkSummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
PyTorch Fundamentals
Installing PyTorchPyTorch tensorsInitializing a tensorOperations on tensorsAuto gradients of tensor objectsAdvantages of PyTorch’s tensors over NumPy’s ndarraysBuilding a neural network using PyTorchDataset, DataLoader, and batch sizePredicting on new data pointsImplementing a custom loss functionFetching the values of intermediate layersUsing a sequential method to build a neural networkSaving and loading a PyTorch modelUsing state_dictSavingLoadingSummaryQuestions
Building a Deep Neural Network with PyTorch
Representing an imageConverting images into structured arrays and scalarsCreating a structured array for colored imagesWhy leverage neural networks for image analysis?Preparing our data for image classificationTraining a neural networkScaling a dataset to improve model accuracyUnderstanding the impact of varying the batch sizeBatch size of 32Batch size of 10,000Understanding the impact of varying the loss optimizerBuilding a deeper neural networkUnderstanding the impact of batch normalizationVery small input values without batch normalizationVery small input values with batch normalizationThe concept of overfittingImpact of adding dropoutImpact of regularizationL1 regularizationL2 regularizationSummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
Section 2: Object Classification and Detection
Introducing Convolutional Neural Networks
The problem with traditional deep neural networksBuilding blocks of a CNNConvolutionFiltersStrides and paddingStridesPaddingPoolingPutting them all togetherHow convolution and pooling help in image translationImplementing a CNNClassifying images using deep CNNsVisualizing the outcome of feature learningBuilding a CNN for classifying real-world imagesImpact on the number of images used for trainingSummaryQuestions
Transfer Learning for Image Classification
Introducing transfer learningUnderstanding the VGG16 architectureImplementing VGG16Understanding the ResNet architectureImplementing ResNet18Implementing facial keypoint detection2D and 3D facial keypoint detectionImplementing age estimation and gender classificationIntroducing the torch_snippets librarySummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
Practical Aspects of Image Classification
Generating CAMsUnderstanding the impact of data augmentation and batch normalizationCoding up road sign detectionPractical aspects to take care of during model implementationImbalanced dataThe size of the object within an imageThe difference between training and validation dataThe number of nodes in the flatten layerImage sizeOpenCV utilitiesSummaryQuestions
Basics of Object Detection
Introducing object detectionCreating a bounding-box ground truth for trainingUnderstanding region proposalsLeveraging SelectiveSearch to generate region proposalsImplementing SelectiveSearch to generate region proposalsUnderstanding IoUNon-max suppressionMean average precisionTraining R-CNN-based custom object detectorsWorking details of R-CNNImplementing R-CNN for object detection on a custom datasetDownloading the datasetPreparing the datasetFetching region proposals and the ground truth of offsetCreating the training dataR-CNN network architecturePredicting on a new imageTraining Fast R-CNN-based custom object detectorsWorking details of Fast R-CNNImplementing Fast R-CNN for object detection on a custom datasetSummaryQuestionsGet This Book’s PDF Version and Exclusive Extras

Advanced Object Detection
Components of modern object detection algorithmsAnchor boxesRegion proposal networkClassification and regressionTraining Faster R-CNN on a custom datasetWorking details of YOLOTraining YOLO on a custom datasetInstalling DarknetSetting up the dataset formatConfiguring the architectureTraining and testing the modelWorking details of SSDComponents in SSD codeSSD300MultiBoxLossTraining SSD on a custom datasetSummaryQuestions
Image Segmentation
Exploring the U-Net architecturePerforming upscalingImplementing semantic segmentation using U-NetExploring the Mask R-CNN architectureRoI AlignMask headImplementing instance segmentation using Mask R-CNNPredicting multiple instances of multiple classesSummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
Applications of Object Detection and Segmentation
Multi-object instance segmentationFetching and preparing dataTraining the model for instance segmentationMaking inferences on a new imageHuman pose detectionCrowd countingImplementing crowd countingImage colorization3D object detection with point cloudsTheoryInput encodingOutput encodingTraining the YOLO model for 3D object detectionData formatData inspectionTrainingTestingAction recognition from videoIdentifying an action in a given videoTraining a recognizer on a custom datasetSummaryQuestions
Section 3: Image Manipulation
Autoencoders and Image Manipulation
Understanding autoencodersHow autoencoders workImplementing vanilla autoencodersImplementing convolutional autoencodersGrouping similar images using t-SNEUnderstanding variational autoencodersThe need for VAEsHow VAEs workKL divergenceBuilding a VAEPerforming an adversarial attack on imagesUnderstanding neural style transferHow neural style transfer worksPerforming neural style transferUnderstanding deepfakesHow deepfakes workGenerating a deepfakeSummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
Image Generation Using GANs
Introducing GANsUsing GANs to generate handwritten digitsUsing DCGANs to generate face imagesImplementing conditional GANsSummaryQuestions
Advanced GANs to Manipulate Images
Leveraging the Pix2Pix GANLeveraging CycleGANHow CycleGAN worksImplementing CycleGANLeveraging StyleGAN on custom imagesThe evolution of StyleGANImplementing StyleGANIntroducing SRGANArchitectureCoding SRGANSummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
Section 4: Combining Computer Vision with Other Techniques
Combining Computer Vision and Reinforcement Learning
Learning the basics of reinforcement learningCalculating the state valueCalculating the state-action valueImplementing Q-learningDefining the Q-valueUnderstanding the Gym environmentBuilding a Q-tableLeveraging exploration-exploitationImplementing deep Q-learningUnderstanding the CartPole environmentPerforming CartPole balancingImplementing deep Q-learning with the fixed targets modelUnderstanding the use caseCoding up an agent to play PongImplementing an agent to perform autonomous drivingSetting up the CARLA environmentInstalling the CARLA binariesInstalling the CARLA Gym environmentTraining a self-driving agentCreating model.pyCreating actor.pyTraining a DQN with fixed targetsSummaryQuestions
Combining Computer Vision and NLP Techniques
Introducing transformersBasics of transformersEncoder blockDecoder blockHow ViTs workImplementing ViTsTranscribing handwritten imagesHandwriting transcription workflowHandwriting transcription in codeDocument layout analysisUnderstanding LayoutLMImplementing LayoutLMv3Visual question answeringIntroducing BLIP2Representation learningGenerative learningImplementing BLIP2SummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
Foundation Models in Computer Vision
Introducing CLIPHow CLIP worksBuilding a CLIP model from scratchLeveraging OpenAI CLIPIntroducing SAMHow SAM worksImplementing SAMHow FastSAM worksAll-instance segmentationPrompt-guided selectionImplementing FastSAMIntroducing diffusion modelsHow diffusion models workDiffusion model architectureImplementing a diffusion model from scratchConditional image generationUnderstanding Stable DiffusionBuilding blocks of the Stable Diffusion modelCrossAttnDownBlock2DUNetMidBlock2DcrossAttnCrossAttnUpBlock2DDownBlock2D, UpBlock2DTransformer2DModelResnetBlock2DImplementing Stable DiffusionSummaryQuestions
Applications of Stable Diffusion
In-paintingModel training workflowIn-painting using Stable DiffusionControlNetArchitectureImplementing ControlNetSDXL TurboArchitectureImplementing SDXL TurboDepthNetWorkflowImplementing DepthNetText to videoWorkflowImplementing text to videoSummaryQuestionsGet This Book’s PDF Version and Exclusive Extras
Moving a Model to Production
Understanding the basics of an APICreating an API and making predictions on a local serverInstalling the API module and dependenciesServing an image classifiersdd.pyserver.pyRunning the serverContainerizing the applicationBuilding a Docker imageCreating the requirements.txt fileCreating a DockerfileBuilding a Docker image and creating a Docker containerShipping and running the Docker container on the cloudConfiguring AWSCreating a Docker repository on AWS ECR and pushing the imagePulling the image and building the Docker containerIdentifying data driftUsing vector storesSummaryQuestions
Appendix
Chapter 1, Artificial Neural Network FundamentalsChapter 2, PyTorch FundamentalsChapter 3, Building a Deep Neural Network with PyTorchChapter 4, Introducing Convolutional Neural NetworksChapter 5, Transfer Learning for Image ClassificationChapter 6, Practical Aspects of Image ClassificationChapter 7, Basics of Object DetectionChapter 8, Advanced Object DetectionChapter 9, Image SegmentationChapter 10, Applications of Object Detection and SegmentationChapter 11, Autoencoders and Image ManipulationChapter 12, Image Generation Using GANsChapter 13, Advanced GANs to Manipulate ImagesChapter 14, Combining Computer Vision and Reinforcement LearningChapter 15, Combining Computer Vision and NLP TechniquesChapter 16, Foundation Models in Computer VisionChapter 17, Applications of Stable DiffusionChapter 18, Moving a Model to Production
Unlock Your Exclusive Benefits
Unlock this Book’s Free Benefits in 3 Easy StepsStep 1Step 2Step 3Need help?
Other Books You May Enjoy
Index

Content preview from Modern Computer Vision with PyTorch - Second Edition

16 Foundation Models in Computer Vision

In the previous chapter, we learned about how we can build novel applications using NLP and CV techniques. However, this requires a significant amount of training, either from scratch or by fine-tuning a pre-trained model. When leveraging a pre-trained model, the model has generally been trained on a large corpus of data – for example, a dataset like ImageNet, which contains ~21 million images. However, on the internet, we have access to hundreds of millions of images and the alt text corresponding to those images. What if we pre-train models on internet-scale data and use those models for different applications involving object detection, segmentation, and text-to-image generation out of the box without ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781803231334

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Modern Computer Vision with PyTorch - Second Edition

by V Kishore Ayyadevara, Yeshwanth Reddy

16

Foundation Models in Computer Vision

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.