Applied Deep Learning and Computer Vision for Self-Driving Cars

Applied Deep Learning and Computer Vision for Self-Driving Cars

by Sumit Ranjan, Dr. S. Senthamilarasu
Released August 2020
Publisher(s): Packt Publishing
ISBN: 9781838646301

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Explore self-driving car technology using deep learning and artificial intelligence techniques and libraries such as TensorFlow, Keras, and OpenCV

Key Features

  • Build and train powerful neural network models to build an autonomous car
  • Implement computer vision, deep learning, and AI techniques to create automotive algorithms
  • Overcome the challenges faced while automating different aspects of driving using modern Python libraries and architectures

Book Description

Thanks to a number of recent breakthroughs, self-driving car technology is now an emerging subject in the field of artificial intelligence and has shifted data scientists' focus to building autonomous cars that will transform the automotive industry. This book is a comprehensive guide to use deep learning and computer vision techniques to develop autonomous cars.

Starting with the basics of self-driving cars (SDCs), this book will take you through the deep neural network techniques required to get up and running with building your autonomous vehicle. Once you are comfortable with the basics, you'll delve into advanced computer vision techniques and learn how to use deep learning methods to perform a variety of computer vision tasks such as finding lane lines, improving image classification, and so on. You will explore the basic structure and working of a semantic segmentation model and get to grips with detecting cars using semantic segmentation. The book also covers advanced applications such as behavior-cloning and vehicle detection using OpenCV, transfer learning, and deep learning methodologies to train SDCs to mimic human driving.

By the end of this book, you'll have learned how to implement a variety of neural networks to develop your own autonomous vehicle using modern Python libraries.

What you will learn

  • Implement deep neural network from scratch using the Keras library
  • Understand the importance of deep learning in self-driving cars
  • Get to grips with feature extraction techniques in image processing using the OpenCV library
  • Design a software pipeline that detects lane lines in videos
  • Implement a convolutional neural network (CNN) image classifier for traffic signal signs
  • Train and test neural networks for behavioral-cloning by driving a car in a virtual simulator
  • Discover various state-of-the-art semantic segmentation and object detection architectures

Who this book is for

If you are a deep learning engineer, AI researcher, or anyone looking to implement deep learning and computer vision techniques to build self-driving blueprint solutions, this book is for you. Anyone who wants to learn how various automotive-related algorithms are built, will also find this book useful. Python programming experience, along with a basic understanding of deep learning, is necessary to get the most of this book.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Applied Deep Learning and Computer Vision for Self-Driving Cars
  3. About Packt
    1. Why subscribe?
  4. Contributors
    1. About the authors
    2. About the reviewers
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Download the color images
    6. Conventions used
    7. Get in touch
    8. Reviews
  6. Section 1: Deep Learning Foundation and SDC Basics
  7. The Foundation of Self-Driving Cars
    1. Introduction to SDCs
    2. Benefits of SDCs
    3. Advancements in SDCs
    4. Challenges in current deployments 
    5. Building safe systems
    6. The cheapest computer and hardware 
    7. Software programming
    8. Fast internet
    9. Levels of autonomy 
    10. Level 0 – manual cars
    11. Level 1 – driver support
    12. Level 2 – partial automation
    13. Level 3 – conditional automation
    14. Level 4 – high automation
    15. Level 5 – complete automation
    16. Deep learning and computer vision approaches for SDCs
    17. LIDAR and computer vision for SDC vision 
    18. Summary
  8. Dive Deep into Deep Neural Networks
    1. Diving deep into neural networks
    2. Introduction to neurons
    3. Understanding neurons and perceptrons
    4. The workings of ANNs
    5. Understanding activation functions
    6. The threshold function
    7. The sigmoid function
    8. The rectifier linear function
    9. The hyperbolic tangent activation function
    10. The cost function of neural networks
    11. Optimizers
    12. Understanding hyperparameters
    13. Model training-specific hyperparameters
    14. Learning rate
    15. Batch size
    16. Number of epochs
    17. Network architecture-specific hyperparameters 
    18. Number of hidden layers 
    19. Regularization
    20. L1 and L2 regularization
    21. Dropout
    22. Activation functions as hyperparameters
    23. TensorFlow versus Keras
    24. Summary
  9. Implementing a Deep Learning Model Using Keras
    1. Starting work with Keras
    2. Advantages of Keras 
    3. The working principle behind Keras
    4. Building Keras models
    5. The sequential model
    6. The functional model
    7. Types of Keras execution
    8. Keras for deep learning
    9. Building your first deep learning model 
    10. Description of the Auto-Mpg dataset
    11. Importing the data
    12. Splitting the data
    13. Standardizing the data
    14. Building and compiling the model
    15. Training the model
    16. Predicting new, unseen data
    17. Evaluating the model's performance
    18. Saving and loading models
    19. Summary
  10. Section 2: Deep Learning and Computer Vision Techniques for SDC
  11. Computer Vision for Self-Driving Cars
    1. Introduction to computer vision
    2. Challenges in computer vision
    3. Artificial eyes versus human eyes 
    4. Building blocks of an image
    5. Digital representation of an image
    6. Converting images from RGB to grayscale
    7. Road-marking detection 
    8. Detection with the grayscale image
    9. Detection with the RGB image
    10. Challenges in color selection techniques
    11. Color space techniques
    12. Introducing the RGB space
    13. HSV space
    14. Color space manipulation 
    15. Introduction to convolution
    16. Sharpening and blurring
    17. Edge detection and gradient calculation
    18. Introducing Sobel 
    19. Introducing the Laplacian edge detector
    20. Canny edge detection
    21. Image transformation
    22. Affine transformation
    23. Projective transformation
    24. Image rotation 
    25. Image translation
    26. Image resizing 
    27. Perspective transformation
    28. Cropping, dilating, and eroding an image
    29. Masking regions of interest
    30. The Hough transform
    31. Summary
  12. Finding Road Markings Using OpenCV
    1. Finding road markings in an image
    2. Loading the image using OpenCV
    3. Converting the image into grayscale
    4. Smoothing the image
    5. Canny edge detection
    6. Masking the region of interest
    7. Applying bitwise_and
    8. Applying the Hough transform
    9. Optimizing the detected road markings
    10. Detecting road markings in a video
    11. Summary
  13. Improving the Image Classifier with CNN
    1. Images in computer format
    2. The need for CNNs
    3. The intuition behind CNNs
    4. Introducing CNNs
    5. Why 3D layers?
    6. Understanding the convolution layer
    7. Depth, stride, and padding 
    8. Depth
    9. Stride 
    10. Zero-padding
    11. ReLU
    12. Fully connected layers
    13. The softmax function
    14. Introduction to handwritten digit recognition
    15. Problem and aim
    16. Loading the data
    17. Reshaping the data
    18. The transformation of data
    19. One-hot encoding the output
    20. Building and compiling our model
    21. Compiling the model 
    22. Training the model
    23. Validation versus train loss
    24. Validation versus test accuracy
    25. Saving the model
    26. Visualizing the model architecture
    27. Confusion matrix 
    28. The accuracy report
    29. Summary
  14. Road Sign Detection Using Deep Learning
    1. Dataset overview
    2. Dataset structure
    3. Image format
    4. Loading the data
    5. Image exploration
    6. Data preparation
    7. Model training
    8. Model accuracy
    9. Summary
  15. Section 3: Semantic Segmentation for Self-Driving Cars
  16. The Principles and Foundations of Semantic Segmentation
    1. Introduction to semantic segmentation
    2. Understanding the semantic segmentation architecture
    3. Overview of different semantic segmentation architectures
    4. U-Net
    5. SegNet
    6. Encoder
    7. Decoder
    8. PSPNet
    9. DeepLabv3+
    10. E-Net
    11. Summary
  17. Implementing Semantic Segmentation
    1. Semantic segmentation in images
    2. Semantic segmentation in videos
    3. Summary
  18. Section 4: Advanced Implementations
  19. Behavioral Cloning Using Deep Learning
    1. Neural network for regression
    2. Behavior cloning using deep learning
    3. Data collection
    4. Data preparation
    5. Model development
    6. Evaluating the simulator
    7. Summary
  20. Vehicle Detection Using OpenCV and Deep Learning
    1. What makes YOLO different?
    2. The YOLO loss function
    3. The YOLO architecture 
    4. Fast YOLO
    5. YOLO v2
    6. YOLO v3
    7. Implementation of YOLO object detection
    8. Importing the libraries
    9. Processing the image function
    10. The get class function
    11. Draw box function
    12. Detect image function
    13. Detect video function
    14. Importing YOLO
    15. Detecting objects in images
    16. Detecting objects in videos
    17. Summary
  21. Next Steps
    1. SDC sensors
    2. Camera
    3. RADAR
    4. Ultrasonic sensors
    5. Odometric sensors
    6. LIDAR 
    7. Introduction to sensor fusion
    8. Kalman filter
    9. Summary
  22. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Applied Deep Learning and Computer Vision for Self-Driving Cars
  • Author(s): Sumit Ranjan, Dr. S. Senthamilarasu
  • Release date: August 2020
  • Publisher(s): Packt Publishing
  • ISBN: 9781838646301