OpenCV 3 Blueprints

OpenCV 3 Blueprints

by Joseph Howse, Steven Puttemans, Quan Hua, Utkarsh Sinha
Released November 2015
Publisher(s): Packt Publishing
ISBN: 9781784399757

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Expand your knowledge of computer vision by building amazing projects with OpenCV 3

About This Book

  • Build computer vision projects to capture high-quality image data, detect and track objects, process the actions of humans or animals, and much more
  • Discover practical and interesting innovations in computer vision while building atop a mature open-source library, OpenCV 3
  • Familiarize yourself with multiple approaches and theories wherever critical decisions need to be made

Who This Book Is For

This book is ideal for you if you aspire to build computer vision systems that are smarter, faster, more complex, and more practical than the competition. This is an advanced book intended for those who already have some experience in setting up an OpenCV development environment and building applications with OpenCV. You should be comfortable with computer vision concepts, object-oriented programming, graphics programming, IDEs, and the command line.

What You Will Learn

  • Select and configure camera systems to see invisible light, fast motion, and distant objects
  • Build a ?camera trap?, as used by nature photographers, and process photos to create beautiful effects
  • Develop a facial expression recognition system with various feature extraction techniques and machine learning methods
  • Build a panorama Android application using the OpenCV stitching module in C++ with NDK support
  • Optimize your object detection model, make it rotation invariant, and apply scene-specific constraints to make it faster and more robust
  • Create a person identification and registration system based on biometric properties of that person, such as their fingerprint, iris, and face
  • Fuse data from videos and gyroscopes to stabilize videos shot from your mobile phone and create hyperlapse style videos

In Detail

Computer vision is becoming accessible to a large audience of software developers who can leverage mature libraries such as OpenCV. However, as they move beyond their first experiments in computer vision, developers may struggle to ensure that their solutions are sufficiently well optimized, well trained, robust, and adaptive in real-world conditions. With sufficient knowledge of OpenCV, these developers will have enough confidence to go about creating projects in the field of computer vision.

This book will help you tackle increasingly challenging computer vision problems that you may face in your careers. It makes use of OpenCV 3 to work around some interesting projects. Inside these pages, you will find practical and innovative approaches that are battle-tested in the authors' industry experience and research. Each chapter covers the theory and practice of multiple complementary approaches so that you will be able to choose wisely in your future projects. You will also gain insights into the architecture and algorithms that underpin OpenCV's functionality.

We begin by taking a critical look at inputs in order to decide which kinds of light, cameras, lenses, and image formats are best suited to a given purpose. We proceed to consider the finer aspects of computational photography as we build an automated camera to assist nature photographers. You will gain a deep understanding of some of the most widely applicable and reliable techniques in object detection, feature selection, tracking, and even biometric recognition. We will also build Android projects in which we explore the complexities of camera motion: first in panoramic image stitching and then in video stabilization.

By the end of the book, you will have a much richer understanding of imaging, motion, machine learning, and the architecture of computer vision libraries and applications!

Style and approach

This book covers a combination of theory and practice. We examine blueprints for specific projects and discuss the principles behind these blueprints, in detail.

Table of contents

  1. OpenCV 3 Blueprints
    1. Table of Contents
    2. OpenCV 3 Blueprints
    3. Credits
    4. About the Authors
    5. About the Reviewers
    6. www.PacktPub.com
      1. Support files, eBooks, discount offers, and more
        1. Why subscribe?
        2. Free access for Packt account holders
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Basic installation guides
      4. Who this book is for
      5. Conventions
      6. Reader feedback
      7. Customer support
        1. Downloading the example code
        2. Downloading the color images of this book
        3. Errata
        4. Piracy
        5. Questions
    8. 1. Getting the Most out of Your Camera System
      1. Coloring the light
      2. Capturing the subject in the moment
      3. Rounding up the unusual suspects
      4. Supercharging the PlayStation Eye
      5. Supercharging the ASUS Xtion PRO Live and other OpenNI-compliant depth cameras
      6. Supercharging the GS3-U3-23S6M-C and other Point Grey Research cameras
      7. Shopping for glass
      8. Summary
    9. 2. Photographing Nature and Wildlife with an Automated Camera
      1. Planning the camera trap
      2. Controlling a photo camera with gPhoto2
        1. Writing a shell script to unmount camera drives
        2. Setting up and testing gPhoto2
        3. Writing a shell script for exposure bracketing
        4. Writing a Python script to wrap gPhoto2
        5. Finding libgphoto2 and wrappers
      3. Detecting the presence of a photogenic subject
        1. Detecting a moving subject
        2. Detecting a colorful subject
        3. Detecting the face of a mammal
      4. Processing images to show subtle colors and motion
        1. Creating HDR images
        2. Creating time-lapse videos
      5. Further study
      6. Summary
    10. 3. Recognizing Facial Expressions with Machine Learning
      1. Introducing facial expression recognition
        1. Facial expression dataset
      2. Finding the face region in the image
        1. Extracting the face region using a face detection algorithm
        2. Extracting facial landmarks from the face region
          1. Introducing the flandmark library
          2. Downloading and compiling the flandmark library
          3. Detecting facial landmarks with flandmark
          4. Visualizing the landmarks in an image
        3. Extracting the face region
        4. Software usage guide
      3. Feature extraction
        1. Extracting image features from facial component regions
          1. Contributed features
          2. Advanced features
          3. Visualizing key points for each feature type
        2. Computing the distribution of feature representation over k clusters
          1. Clustering image features space into k clusters
          2. Computing a final feature for each image
          3. Dimensionality reduction
        3. Software usage guide
      4. Classification
        1. Classification process
        2. Splitting the dataset into a training set and testing set
        3. Support vector machines
          1. Training stage
          2. Testing stage
        4. Multi-layer perceptron
          1. Training stage
            1. Define the network
            2. Train the network
          2. Testing stage
        5. K-Nearest Neighbors (KNN)
          1. Training stage
          2. The testing stage
        6. Normal Bayes classifier
          1. Training stage
          2. Testing stage
        7. Software usage guide
      5. Evaluation
        1. Evaluation with different learning algorithms
        2. Evaluation with different features
        3. Evaluation with a different number of clusters
      6. System overview
      7. Further reading
        1. Compiling the opencv_contrib module
        2. Kaggle facial expression dataset
        3. Facial landmarks
          1. What are facial landmarks?
          2. How do you detect facial landmarks?
          3. How do you use facial landmarks?
        4. Improving feature extraction
        5. K-fold cross validation
      8. Summary
    11. 4. Panoramic Image Stitching Application Using Android Studio and NDK
      1. Introducing the concept of panorama
      2. The Android section – an application user interface
        1. The setup activity layout
        2. Capturing the camera frame
          1. Using the Camera API to get the camera frame
        3. Implementing the Capture button
        4. Implementing the Save button
      3. Integrating OpenCV into the Android Studio
        1. Compiling OpenCV Android SDK to the Android Studio project
        2. Setting up the Android Studio to work with OpenCV
          1. Importing the OpenCV Android SDK
          2. Creating a Java and C++ interaction with Java Native Interface (JNI)
          3. Compiling OpenCV C++ with NDK/JNI
        3. Implementing the OpenCV Java code
        4. Implementing the OpenCV C++ code
      4. Application showcase
      5. Further improvement
      6. Summary
    12. 5. Generic Object Detection for Industrial Applications
      1. Difference between recognition, detection, and categorization
      2. Smartly selecting and preparing application specific training data
        1. The amount of training data
        2. Creating object annotation files for the positive samples
        3. Parsing your positive dataset into the OpenCV data vector
      3. Parameter selection when training an object model
        1. Training parameters involved in training an object model
        2. The cascade classification process in detail
          1. Step 1 – grabbing positive and negative samples
          2. Step 2 – precalculation of integral image and all possible features from the training data
          3. Step 3 – firing up the boosting process
          4. Step 4 – saving the temporary result to a stage file
        3. The resulting object model explained in detail
          1. HAAR-like wavelet feature models
          2. Local binary pattern models
          3. Visualization tool for object models
        4. Using cross-validation to achieve the best model possible
      4. Using scene specific knowledge and constraints to optimize the detection result
        1. Using the parameters of the detection command to influence your detection result
        2. Increasing object instance detection and reducing false positive detections
      5. Obtaining rotation invariance object detection
      6. 2D scale space relation
      7. Performance evaluation and GPU optimizations
        1. Object detection performance testing
        2. Optimizations using GPU code
      8. Practical applications
      9. Summary
    13. 6. Efficient Person Identification Using Biometric Properties
      1. Biometrics, a general approach
        1. Step 1 – getting a good training dataset and applying application-specific normalization
        2. Step 2 – creating a descriptor of the recorded biometric
        3. Step 3 – using machine learning to match the retrieved feature vector
        4. Step 4 – think about your authentication process
      2. Face detection and recognition
        1. Face detection using the Viola and Jones boosted cascade classifier algorithm
        2. Data normalization on the detected face regions
          1. Various face recognition approaches and their corresponding feature space
          2. Eigenface decomposition through PCA
          3. Linear discriminant analysis using the Fisher criterion
          4. Local binary pattern histograms
        3. The problems with facial recognition in its current OpenCV 3 based implementation
      3. Fingerprint identification, how is it done?
        1. Implementing the approach in OpenCV 3
      4. Iris identification, how is it done?
        1. Implementing the approach in OpenCV 3
      5. Combining the techniques to create an efficient people-registration system
      6. Summary
    14. 7. Gyroscopic Video Stabilization
      1. Stabilization with images
      2. Stabilization with hardware
      3. A hybrid of hardware and software
      4. The math
        1. The camera model
        2. The Camera motion
        3. Rolling shutter compensation
        4. Image warping
      5. Project overview
      6. Capturing data
        1. Recording video
        2. Recording gyro signals
      7. Android specifics
        1. Threaded overlay
        2. Reading media files
      8. Calibration
        1. Data structures
          1. Reading the gyroscope trace
          2. The training video
        2. Handling rotations
          1. Rotating an image
          2. Accumulated rotations
        3. The calibration class
        4. Undistorting images
        5. Testing calibration results
      9. Rolling shutter compensation
        1. Calibrating the rolling shutter
        2. Warping with grid points
        3. Unwarping with calibration
      10. What's next?
        1. Identifying gyroscope axes
        2. Estimating the rolling shutter direction
        3. Smoother timelapses
        4. Repository of calibration parameters
        5. Incorporating translations
        6. Additional tips
          1. Use the Python pickle module
          2. Write out single images
          3. Testing without the delta
      11. Summary
    15. Index

Product information

  • Title: OpenCV 3 Blueprints
  • Author(s): Joseph Howse, Steven Puttemans, Quan Hua, Utkarsh Sinha
  • Release date: November 2015
  • Publisher(s): Packt Publishing
  • ISBN: 9781784399757