Building Computer Vision Projects with OpenCV 4 and C++

Building Computer Vision Projects with OpenCV 4 and C++

by David Millan Escriva, Prateek Joshi, Vinicius G. Mendonca, Roy Shilkrot
Released March 2019
Publisher(s): Packt Publishing
ISBN: 9781838644673

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Delve into practical computer vision and image processing projects and get up to speed with advanced object detection techniques and machine learning algorithms

Key Features

  • Discover best practices for engineering and maintaining OpenCV projects
  • Explore important deep learning tools for image classification
  • Understand basic image matrix formats and filters

Book Description

OpenCV is one of the best open source libraries available and can help you focus on constructing complete projects on image processing, motion detection, and image segmentation.

This Learning Path is your guide to understanding OpenCV concepts and algorithms through real-world examples and activities. Through various projects, you'll also discover how to use complex computer vision and machine learning algorithms and face detection to extract the maximum amount of information from images and videos. In later chapters, you'll learn to enhance your videos and images with optical flow analysis and background subtraction. Sections in the Learning Path will help you get to grips with text segmentation and recognition, in addition to guiding you through the basics of the new and improved deep learning modules. By the end of this Learning Path, you will have mastered commonly used computer vision techniques to build OpenCV projects from scratch. This Learning Path includes content from the following Packt books:

  • Mastering OpenCV 4 - Third Edition by Roy Shilkrot and David Millan Escriva
  • Learn OpenCV 4 By Building Projects - Second Edition by David Millan Escriva, Vinicius G. Mendonca, and Prateek Joshi

What you will learn

  • Stay up-to-date with algorithmic design approaches for complex computer vision tasks
  • Work with OpenCV's most up-to-date API through various projects
  • Understand 3D scene reconstruction and Structure from Motion (SfM)
  • Study camera calibration and overlay augmented reality (AR) using the ArUco module
  • Create CMake scripts to compile your C++ application
  • Explore segmentation and feature extraction techniques
  • Remove backgrounds from static scenes to identify moving objects for surveillance
  • Work with new OpenCV functions to detect and recognize text with Tesseract

Who this book is for

If you are a software developer with a basic understanding of computer vision and image processing and want to develop interesting computer vision applications with OpenCV, this Learning Path is for you. Prior knowledge of C++ and familiarity with mathematical concepts will help you better understand the concepts in this Learning Path.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Building Computer Vision Projects with OpenCV 4 and C++
  3. About Packt
    1. Why subscribe?
    2. Packt.com
  4. Contributors
    1. About the authors
    2. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  6. Getting Started with OpenCV
    1. Understanding the human visual system
    2. How do humans understand image content?
      1. Why is it difficult for machines to understand image content?
    3. What can you do with OpenCV?
      1. Inbuilt data structures and input/output
      2. Image processing operations
      3. GUI
      4. Video analysis
      5. 3D reconstruction
      6. Feature extraction
      7. Object detection
      8. Machine learning
      9. Computational photography
      10. Shape analysis
      11. Optical flow algorithms
      12. Face and object recognition
      13. Surface matching
      14. Text detection and recognition
      15. Deep learning
    4. Installing OpenCV
      1. Windows
      2. Mac OS X
      3. Linux
    5. Summary
  7. An Introduction to the Basics of OpenCV
    1. Technical requirements
    2. Basic CMake configuration file
    3. Creating a library
    4. Managing dependencies
    5. Making the script more complex
    6. Images and matrices
    7. Reading/writing images
    8. Reading videos and cameras
    9. Other basic object types
      1. Vec object type
      2. Scalar object type
      3. Point object type
      4. Size object type
      5. Rect object type
      6. RotatedRect object type
    10. Basic matrix operations
    11. Basic data persistence and storage
      1. Writing to FileStorage
    12. Summary
  8. Learning Graphical User Interfaces
    1. Technical requirements
    2. Introducing the OpenCV user interface
    3. Basic graphical user interface with OpenCV
      1. Adding slider and mouse events to our interfaces
    4. Graphic user interface with Qt
      1. Adding buttons to the user interface
    5. OpenGL support
    6. Summary
  9. Delving into Histogram and Filters
    1. Technical requirements
    2. Generating a CMake script file
    3. Creating the graphical user interface
    4. Drawing a histogram
    5. Image color equalization
    6. Lomography effect
    7. Cartoonize effect
    8. Summary
  10. Automated Optical Inspection, Object Segmentation, and Detection
    1. Technical requirements
    2. Isolating objects in a scene
    3. Creating an application for AOI
    4. Preprocessing the input image
      1. Noise removal
      2. Removing the background using the light pattern for segmentation
      3. Thresholding
    5. Segmenting our input image
      1. The connected components algorithm
      2. The findContours algorithm
    6. Summary
  11. Learning Object Classification
    1. Technical requirements
    2. Introducing machine learning concepts
      1. OpenCV machine learning algorithms
    3. Computer vision and the machine learning workflow 
    4. Automatic object inspection classification example
      1. Feature extraction
      2. Training an SVM model
      3. Input image prediction
    5. Summary
  12. Detecting Face Parts and Overlaying Masks
    1. Technical requirements
    2. Understanding Haar cascades
    3. What are integral images?
    4. Overlaying a face mask in a live video
      1. What happened in the code?
    5. Get your sunglasses on
      1. Looking inside the code
    6. Tracking the nose, mouth, and ears
    7. Summary
  13. Video Surveillance, Background Modeling, and Morphological Operations
    1. Technical requirements
    2. Understanding background subtraction
    3. Naive background subtraction
      1. Does it work well?
    4. Frame differencing
      1. How well does it work?
    5. The Mixture of Gaussians approach
      1. What happened in the code?
    6. Morphological image processing
      1. What's the underlying principle?
    7. Slimming the shapes
    8. Thickening the shapes
    9. Other morphological operators
      1. Morphological opening
      2. Morphological closing
      3. Drawing the boundary
      4. Top Hat transform
      5. Black Hat transform
    10. Summary
  14. Learning Object Tracking
    1. Technical requirements
    2. Tracking objects of a specific color
    3. Building an interactive object tracker
    4. Detecting points using the Harris corner detector
    5. Good features to track
    6. Feature-based tracking
      1. Lucas-Kanade method
      2. Farneback algorithm
    7. Summary
  15. Developing Segmentation Algorithms for Text Recognition
    1. Technical requirements
    2. Introducing optical character recognition
    3. Preprocessing stage
      1. Thresholding the image
      2. Text segmentation
        1. Creating connected areas
        2. Identifying paragraph blocks
        3. Text extraction and skewing adjustment
    4. Installing Tesseract OCR on your operating system
      1. Installing Tesseract on Windows
        1. Building the latest library
        2. Setting up Tesseract in Visual Studio
          1. Static linking
      2. Installing Tesseract on Mac
    5. Using the Tesseract OCR library
      1. Creating an OCR function
        1. Sending the output to a file
    6. Summary
  16. Text Recognition with Tesseract
    1. Technical requirements
    2. How the text API works
      1. The scene detection problem
      2. Extremal regions
      3. Extremal region filtering
    3. Using the text API
      1. Text detection
      2. Text extraction
      3. Text recognition
    4. Summary
  17. Deep Learning with OpenCV
    1. Technical requirements
    2. Introduction to deep learning
      1. What is a neural network and how can we learn from data?
      2. Convolutional neural networks
    3. Deep learning in OpenCV
    4. YOLO – real-time object detection
      1. YOLO v3 deep learning model architecture
      2. The YOLO dataset, vocabulary, and model
      3. Importing YOLO into OpenCV
    5. Face detection with SSD
      1. SSD model architecture
      2. Importing SSD face detection into OpenCV
    6. Summary
  18. Cartoonifier and Skin Color Analysis on the RaspberryPi
    1. Accessing the webcam
    2. Main camera processing loop for a desktop app
      1. Generating a black and white sketch
      2. Generating a color painting and a cartoon
      3. Generating an evil mode using edge filters
      4. Generating an alien mode using skin detection
        1. Skin detection algorithm
        2. Showing the user where to put their face
    3. Implementation of the skin color changer
      1. Reducing the random pepper noise from the sketch image
    4. Porting from desktop to an embedded device
      1. Equipment setup to develop code for an embedded device
        1. Configuring a new Raspberry Pi
      2. Installing OpenCV on an embedded device
        1. Using the Raspberry Pi Camera Module
          1. Installing the Raspberry Pi Camera Module driver
        2. Making Cartoonifier run in fullscreen
        3. Hiding the mouse cursor
        4. Running Cartoonifier automatically after bootup
        5. Speed comparison of Cartoonifier on desktop versus embedded
          1. Changing the camera and camera resolution
        6. Power draw of Cartoonifier running on desktop versus embedded system
          1. Streaming video from Raspberry Pi to a powerful computer
        7. Customizing your embedded system!
    5. Summary
  19. Explore Structure from Motion with the SfM Module
    1. Technical requirements
    2. Core concepts of SfM
      1. Calibrated cameras and epipolar geometry
      2. Stereo reconstruction and SfM
    3. Implementing SfM in OpenCV
      1. Image feature matching
      2. Finding feature tracks
      3. 3D reconstruction and visualization
      4. MVS for dense reconstruction
    4. Summary
  20. Face Landmark and Pose with the Face Module
    1. Technical requirements
    2. Theory and context
      1. Active appearance models and constrained local models
      2. Regression methods
    3. Facial landmark detection in OpenCV
      1. Measuring error
    4. Estimating face direction from landmarks
      1. Estimated pose calculation
      2. Projecting the pose on the image
    5. Summary
  21. Number Plate Recognition with Deep Convolutional Networks
    1. Introduction to ANPR
    2. ANPR algorithm
    3. Plate detection
      1. Segmentation
      2. Classification
    4. Plate recognition
      1. OCR segmentation
      2. Character classification using a convolutional neural network
        1. Creating and training a convolutional neural network with TensorFlow
          1. Preparing the data
          2. Creating a TensorFlow model
          3. Preparing a model for OpenCV
          4. Import and use model in OpenCV C++ code
    5. Summary
  22. Face Detection and Recognition with the DNN Module
    1. Introduction to face detection and face recognition
      1. Face detection
        1. Implementing face detection using OpenCV cascade classifiers
          1. Loading a Haar or LBP detector for object or face detection
          2. Accessing the webcam
          3. Detecting an object using the Haar or LBP classifier
          4. Detecting the face
        2. Implementing face detection using the OpenCV deep learning module
      2. Face preprocessing
        1. Eye detection
        2. Eye search regions
          1. Geometrical transformation
          2. Separate histogram equalization for left and right sides
          3. Smoothing
          4. Elliptical mask
      3. Collecting faces and learning from them
        1. Collecting preprocessed faces for training
        2. Training the face recognition system from collected faces
        3. Viewing the learned knowledge
        4. Average face
        5. Eigenvalues, Eigenfaces, and Fisherfaces
      4. Face recognition
        1. Face identification – recognizing people from their faces
        2. Face verification—validating that it is the claimed person
      5. Finishing touches—saving and loading files
      6. Finishing touches—making a nice and interactive GUI
        1. Drawing the GUI elements
          1. Startup mode
          2. Detection mode
          3. Collection mode
          4. Training mode
          5. Recognition mode
        2. Checking and handling mouse clicks
    2. Summary
    3. References
  23. Android Camera Calibration and AR Using the ArUco Module
    1. Technical requirements
    2. Augmented reality and pose estimation
      1. Camera calibration
      2. Augmented reality markers for planar reconstruction
    3. Camera access in Android OS
      1. Finding and opening the camera
    4. Camera calibration with ArUco
    5. Augmented reality with jMonkeyEngine
    6. Summary
  24. iOS Panoramas with the Stitching Module
    1. Technical requirements
    2. Panoramic image stitching methods
      1. Feature extraction and robust matching for panoramas
        1. Affine constraint
        2. Random sample consensus (RANSAC)
        3. Homography constraint
        4. Bundle Adjustment
      2. Warping images for panorama creation
    3. Project overview
    4. Setting up an iOS OpenCV project with CocoaPods 
    5. iOS UI for panorama capture
    6. OpenCV stitching in an Objective-C++ wrapper
    7. Summary
    8. Further reading
  25. Finding the Best OpenCV Algorithm for the Job
    1. Technical requirements
    2. Is it covered in OpenCV?
    3. Algorithm options in OpenCV
    4. Which algorithm is best?
    5. Example comparative performance test of algorithms
    6. Summary
  26. Avoiding Common Pitfalls in OpenCV
    1. History of OpenCV from v1 to v4
      1. OpenCV and the data revolution in computer vision
    2. Historic algorithms in OpenCV
      1. How to check when an algorithm was added to OpenCV
    3. Common pitfalls and suggested solutions
    4. Summary
    5. Further reading
  27. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Building Computer Vision Projects with OpenCV 4 and C++
  • Author(s): David Millan Escriva, Prateek Joshi, Vinicius G. Mendonca, Roy Shilkrot
  • Release date: March 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781838644673