Hands-On Computer Vision with Detectron2

Hands-On Computer Vision with Detectron2

by Van Vung Pham, Tommy Dang
Released April 2023
Publisher(s): Packt Publishing
ISBN: 9781800561625

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Explore Detectron2 using cutting-edge models and learn all about implementing future computer vision applications in custom domains

Purchase of the print or Kindle book includes a free PDF eBook

Key Features

  • Learn how to tackle common computer vision tasks in modern businesses with Detectron2
  • Leverage Detectron2 performance tuning techniques to control the model's finest details
  • Deploy Detectron2 models into production and develop Detectron2 models for mobile devices

Book Description

Computer vision is a crucial component of many modern businesses, including automobiles, robotics, and manufacturing, and its market is growing rapidly. This book helps you explore Detectron2, Facebook's next-gen library providing cutting-edge detection and segmentation algorithms. It's used in research and practical projects at Facebook to support computer vision tasks, and its models can be exported to TorchScript or ONNX for deployment.

The book provides you with step-by-step guidance on using existing models in Detectron2 for computer vision tasks (object detection, instance segmentation, key-point detection, semantic detection, and panoptic segmentation). You'll get to grips with the theories and visualizations of Detectron2's architecture and learn how each module in Detectron2 works. As you advance, you'll build your practical skills by working on two real-life projects (preparing data, training models, fine-tuning models, and deployments) for object detection and instance segmentation tasks using Detectron2. Finally, you'll deploy Detectron2 models into production and develop Detectron2 applications for mobile devices.

By the end of this deep learning book, you'll have gained sound theoretical knowledge and useful hands-on skills to help you solve advanced computer vision tasks using Detectron2.

What you will learn

  • Build computer vision applications using existing models in Detectron2
  • Grasp the concepts underlying Detectron2's architecture and components
  • Develop real-life projects for object detection and object segmentation using Detectron2
  • Improve model accuracy using Detectron2's performance-tuning techniques
  • Deploy Detectron2 models into server environments with ease
  • Develop and deploy Detectron2 models into browser and mobile environments

Who this book is for

If you are a deep learning application developer, researcher, or software developer with some prior knowledge about deep learning, this book is for you to get started and develop deep learning models for computer vision applications. Even if you are an expert in computer vision and curious about the features of Detectron2, or you would like to learn some cutting-edge deep learning design patterns, you will find this book helpful. Some HTML, Android, and C++ programming skills are advantageous if you want to deploy computer vision applications using these platforms.

Table of contents

  1. Hands-On Computer Vision with Detectron2
  2. Foreword
  3. Contributors
  4. About the author
  5. About the reviewers
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Code in Action
    6. Conventions used
    7. Get in touch
    8. Share Your Thoughts
    9. Download a free PDF copy of this book
  7. Part 1: Introduction to Detectron2
  8. Chapter 1: An Introduction to Detectron2 and Computer Vision Tasks
    1. Technical requirements
    2. Computer vision tasks
      1. Object detection
      2. Instance segmentation
      3. Keypoint detection
      4. Semantic segmentation
      5. Panoptic segmentation
    3. An introduction to Detectron2 and its architecture
      1. Introducing Detectron2
      2. Detectron2 architecture
    4. Detectron2 development environments
      1. Cloud development environment for Detectron2 applications
      2. Local development environment for Detectron2 applications
      3. Connecting Google Colab to a local development environment
    5. Summary
  9. Chapter 2: Developing Computer Vision Applications Using Existing Detectron2 Models
    1. Technical requirements
    2. Introduction to Detectron2’s Model Zoo
    3. Developing an object detection application
      1. Getting the configuration file
      2. Getting a predictor
      3. Performing inferences
      4. Visualizing the results
    4. Developing an instance segmentation application
      1. Selecting a configuration file
      2. Getting a predictor
      3. Performing inferences
      4. Visualizing the results
    5. Developing a keypoint detection application
      1. Selecting a configuration file
      2. Getting a predictor
      3. Performing inferences
      4. Visualizing the results
    6. Developing a panoptic segmentation application
      1. Selecting a configuration file
      2. Getting a predictor
      3. Performing inferences
      4. Visualizing the results
    7. Developing a semantic segmentation application
      1. Selecting a configuration file and getting a predictor
      2. Performing inferences
      3. Visualizing the results
    8. Putting it all together
      1. Getting a predictor
      2. Performing inferences
      3. Visualizing the results
      4. Performing a computer vision task
    9. Summary
  10. Part 2: Developing Custom Object Detection Models
  11. Chapter 3: Data Preparation for Object Detection Applications
    1. Technical requirements
    2. Common data sources
    3. Getting images
    4. Selecting an image labeling tool
    5. Annotation formats
    6. Labeling the images
    7. Annotation format conversions
      1. Converting YOLO datasets to COCO datasets
      2. Converting Pascal VOC datasets to COCO datasets
    8. Summary
  12. Chapter 4: The Architecture of the Object Detection Model in Detectron2
    1. Technical requirements
    2. Introduction to the application architecture
    3. The backbone network
    4. Region Proposal Network
      1. The anchor generator
      2. The RPN head
      3. The RPN loss calculation
      4. Proposal predictions
    5. Region of Interest Heads
      1. The pooler
      2. The box predictor
    6. Summary
  13. Chapter 5: Training Custom Object Detection Models
    1. Technical requirements
    2. Processing data
      1. The dataset
      2. Downloading and performing initial explorations
      3. Data format conversion
      4. Displaying samples
    3. Using the default trainer
    4. Selecting the best model
      1. Evaluation metrics for object detection models
      2. Selecting the best model
      3. Inferencing thresholds
      4. Sample predictions
    5. Developing a custom trainer
    6. Utilizing the hook system
    7. Summary
  14. Chapter 6: Inspecting Training Results and Fine-Tuning Detectron2’s Solvers
    1. Technical requirements
    2. Inspecting training histories with TensorBoard
    3. Understanding Detectron2’s solvers
      1. Gradient descent
      2. Stochastic gradient descent
      3. Momentum
      4. Variable learning rates
    4. Fine-tuning the learning rate and batch size
    5. Summary
  15. Chapter 7: Fine-Tuning Object Detection Models
    1. Technical requirements
    2. Setting anchor sizes and anchor ratios
      1. Preprocessing input images
      2. Sampling training data and generating the default anchors
      3. Generating sizes and ratios hyperparameters
    3. Setting pixel means and standard deviations
      1. Preparing a data loader
      2. Calculating the running means and standard deviations
    4. Putting it all together
    5. Summary
  16. Chapter 8: Image Data Augmentation Techniques
    1. Technical requirements
    2. Image augmentation techniques
      1. Why image augmentations?
      2. What are image augmentations?
      3. How to perform image augmentations
    3. Detectron2’s image augmentation system
      1. Transformation classes
      2. Augmentation classes
      3. The AugInput class
    4. Summary
  17. Chapter 9: Applying Train-Time and Test-Time Image Augmentations
    1. Technical requirements
    2. The Detectron2 data loader
    3. Applying existing image augmentation techniques
    4. Developing custom image augmentation techniques
      1. Modifying the existing data loader
      2. Developing the MixUp image augmentation technique
      3. Developing the Mosaic image augmentation technique
    5. Applying test-time image augmentation techniques
    6. Summary
  18. Part 3: Developing a Custom Detectron2 Model for Instance Segmentation Tasks
  19. Chapter 10: Training Instance Segmentation Models
    1. Technical requirements
    2. Preparing data for training segmentation models
      1. Getting images, labeling images, and converting annotations
      2. Introduction to the brain tumor segmentation dataset
    3. The architecture of the segmentation models
    4. Training custom segmentation models
    5. Summary
  20. Chapter 11: Fine-Tuning Instance Segmentation Models
    1. Technical requirements
    2. Introduction to PointRend
    3. Using existing PointRend models
    4. Training custom PointRend models
    5. Summary
  21. Part 4: Deploying Detectron2 Models into Production
  22. Chapter 12: Deploying Detectron2 Models into Server Environments
    1. Technical requirements
    2. Supported file formats and runtimes
      1. Development environments, file formats, and runtimes
      2. Exporting PyTorch models using the tracing method
      3. When the tracing method fails
      4. Exporting PyTorch models using the scripting method
      5. Mixing tracing and scripting approaches
      6. Deploying models using a C++ environment
    3. Deploying custom Detectron2 models
      1. Detectron2 utilities for exporting models
      2. Exporting a custom Detectron2 model
    4. Summary
  23. Chapter 13: Deploying Detectron2 Models into Browsers and Mobile Environments
    1. Technical requirements
    2. Deploying Detectron2 models using ONNX
      1. Introduction to ONNX
      2. Exporting a PyTorch model to ONNX
      3. Loading an ONNX model to the browser
      4. Exporting a custom Detectron2 model to ONNX
    3. Developing mobile computer vision apps with D2Go
      1. Introduction to D2Go
      2. Using existing D2Go models
      3. Training custom D2Go models
      4. Model quantization
    4. Summary
  24. Index
    1. Why subscribe?
  25. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts
    3. Download a free PDF copy of this book

Product information

  • Title: Hands-On Computer Vision with Detectron2
  • Author(s): Van Vung Pham, Tommy Dang
  • Release date: April 2023
  • Publisher(s): Packt Publishing
  • ISBN: 9781800561625