Artificial Vision and Language Processing for Robotics

Artificial Vision and Language Processing for Robotics

by Alvaro Morena Alberola, Gonzalo Molina Gallego, Unai Garay Maestre
Released April 2019
Publisher(s): Packt Publishing
ISBN: 9781838552268

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Create end-to-end systems that can power robots with artificial vision and deep learning techniques

Key Features

  • Study ROS, the main development framework for robotics, in detail
  • Learn all about convolutional neural networks, recurrent neural networks, and robotics
  • Create a chatbot to interact with the robot

Book Description

Artificial Vision and Language Processing for Robotics begins by discussing the theory behind robots. You'll compare different methods used to work with robots and explore computer vision, its algorithms, and limits. You'll then learn how to control the robot with natural language processing commands. You'll study Word2Vec and GloVe embedding techniques, non-numeric data, recurrent neural network (RNNs), and their advanced models. You'll create a simple Word2Vec model with Keras, as well as build a convolutional neural network (CNN) and improve it with data augmentation and transfer learning. You'll study the ROS and build a conversational agent to manage your robot. You'll also integrate your agent with the ROS and convert an image to text and text to speech. You'll learn to build an object recognition system using a video.

By the end of this book, you'll have the skills you need to build a functional application that can integrate with a ROS to extract useful information about your environment.

What you will learn

  • Explore the ROS and build a basic robotic system
  • Understand the architecture of neural networks
  • Identify conversation intents with NLP techniques
  • Learn and use the embedding with Word2Vec and GloVe
  • Build a basic CNN and improve it using generative models
  • Use deep learning to implement artificial intelligence(AI)and object recognition
  • Develop a simple object recognition system using CNNs
  • Integrate AI with ROS to enable your robot to recognize objects

Who this book is for

Artificial Vision and Language Processing for Robotics is for robotics engineers who want to learn how to integrate computer vision and deep learning techniques to create complete robotic systems. It will prove beneficial to you if you have working knowledge of Python and a background in deep learning. Knowledge of the ROS is a plus.

Table of contents

  1. Preface
    1. About the Book
      1. About the Authors
      2. Objectives
      3. Audience
      4. Approach
      5. Minimum Hardware Requirements
      6. Software Requirements
      7. Conventions
      8. Installation and Setup
      9. Additional Resources
  2. Chapter 1
  3. Fundamentals of Robotics
    1. Introduction
    2. History of Robotics
    3. Artificial Intelligence
      1. Natural Language Processing
      2. Computer Vision
      3. Types of Robots
      4. Industrial Robots
      5. Service Robots
      6. Hardware and Software of Robots
    4. Robot Positioning
      1. Exercise 1: Computing a Robot’s Position
      2. How to Work with Robots
      3. Exercise 2: Computing the Distance Traveled by a Wheel with Python
      4. Exercise 3: Computing Final Position with Python
      5. Activity 1: Robot Positioning Using Odometry with Python
    5. Summary
  4. Chapter 2
  5. Introduction to Computer Vision
    1. Introduction
    2. Basic Algorithms in Computer Vision
      1. Image Terminology
      2. OpenCV
      3. Basic Image Processing Algorithms
      4. Thresholding
      5. Exercise 4: Applying Various Thresholds to an Image
      6. Morphological Transformations
      7. Exercise 5: Applying the Various Morphological Transformations to an Image
      8. Blurring (Smoothing)
      9. Exercise 6: Applying the Various Blurring Methods to an Image
      10. Exercise 7: Loading an Image and Applying the Learned Methods
    3. Introduction to Machine Learning
      1. Decision Trees and Boosting Algorithms
      2. Bagging:
      3. Boosting
      4. Exercise 8: Predicting Numbers Using the Decision Tree, Random Forest, and AdaBoost Algorithms
      5. Artificial Neural Networks (ANNs)
      6. Exercise 9: Building Your First Neural Network
      7. Activity 2: Classify 10 Types of Clothes from the Fashion-MNIST Database
    4. Summary
  6. Chapter 3
  7. Fundamentals of Natural Language Processing
    1. Introduction
      1. Natural Language Processing
      2. Parts of NLP
      3. Levels of NLP
    2. NLP in Python
      1. Natural Language Toolkit (NLTK)
      2. Exercise 10: Introduction to NLTK
      3. spaCy
      4. Exercise 11: Introduction to spaCy
    3. Topic Modeling
      1. Term Frequency – Inverse Document Frequency (TF-IDF)
      2. Latent Semantic Analysis (LSA)
      3. Exercise 12: Topic Modeling in Python
      4. Activity 3: Process a Corpus
    4. Language Modeling
      1. Introduction to Language Models
      2. The Bigram Model
      3. N-gram Model
      4. Calculating Probabilities
      5. Exercise 13: Create a Bigram Model
    5. Summary
  8. Chapter 4
  9. Neural Networks with NLP
    1. Introduction
    2. Recurrent Neural Networks
      1. Introduction to Recurrent Neural Networks (RNN)
      2. Inside Recurrent Neural Networks
      3. RNN architectures
      4. Long-Dependency Problem
      5. Exercise 14: Predict House Prices with an RNN
    3. Long Short-Term Memory
      1. Exercise 15: Predict the Next Solution of a Mathematical Function
    4. Neural Language Models
      1. Introduction to Neural Language Models
      2. RNN Language Model
      3. Exercise 16: Encoding a Small Corpus
      4. The Input Dimensions of RNNs
      5. Activity 4: Predict the Next Character in a Sequence
    5. Summary
  10. Chapter 5
  11. Convolutional Neural Networks for Computer Vision
    1. Introduction
    2. Fundamentals of CNNs
    3. Building Your First CNN
      1. Exercise 17: Building a CNN
    4. Improving Your Model - Data Augmentation
      1. Exercise 18: Improving Models Using Data Augmentation
      2. Activity 5: Making Use of Data Augmentation to Classify correctly Images of Flowers
    5. State-of-the-Art Models - Transfer Learning
      1. Exercise 19: Classifying €5 and €20 Bills Using Transfer Learning with Very Little Data
    6. Summary
  12. Chapter 6
  13. Robot Operating System (ROS)
    1. Introduction
    2. ROS Concepts
    3. ROS Commands
    4. Installation and Configuration
    5. Catkin Workspaces and Packages
    6. Publishers and Subscribers
      1. Exercise 20: Publishing and Subscribing
      2. Exercise 21: Publishers and Subscribers
    7. Simulators
      1. Exercise 22: The Turtlebot configuration
      2. Exercise 23: Simulators and Sensors
      3. Activity 6: Simulators and Sensors
    8. Summary
  14. Chapter 7
  15. Build a Text-Based Dialogue System (Chatbot)
    1. Introduction
    2. Word Representation in Vector Space
      1. Word Embeddings
      2. Cosine Similarity
      3. Word2Vec
      4. Problems with Word2Vec
      5. Gensim
      6. Exercise 24: Creation of a Word Embedding
      7. Global Vectors (GloVe)
      8. Exercise 25: Using a Pretrained GloVe to See the Distribution of Words in a Plane
    3. Dialogue Systems
      1. Tools for Developing Chatbots
      2. Types of Conversational Agents
      3. Classification by Input-Output Data Type
      4. Classification by System Knowledge
      5. Creation of a Text-Based Dialogue System
      6. Exercise 26: Create Your First Conversational Agent
      7. Activity 7: Create a Conversational Agent to Control a Robot
    4. Summary
  16. Chapter 8
  17. Object Recognition to Guide a Robot Using CNNs
    1. Introduction
    2. Multiple Object Recognition and Detection
      1. Exercise 24: Building Your First Multiple Object Detection and Recognition Algorithm
      2. ImageAI
    3. Multiple Object Recognition and Detection in Video
      1. Activity 8: Multiple Object Detection and Recognition in Video
    4. Summary
  18. Chapter 9
  19. Computer Vision for Robotics
    1. Introduction
    2. Darknet
      1. Basic Installation of Darknet
    3. YOLO
      1. First Steps in Image Classification with YOLO
      2. YOLO on a Webcam
      3. Exercise 28: Programming with YOLO
      4. ROS Integration
      5. Exercise 29: ROS and YOLO Integration
      6. Activity 9: A Robotic Security Guard
    4. Summary
  20. Appendix
    1. Chapter 1: Fundamentals of Robotics
      1. Activity 1: Robot Positioning Using Odometry with Python
    2. Chapter 2: Introduction to Computer Vision
      1. Activity 2: Classify 10 Types of Clothes from the Fashion-MNIST Data
    3. Chapter 3: Fundamentals of Natural Language Processing
      1. Activity 3: Process a Corpus
    4. Chapter 4: Neural Networks with NLP
      1. Activity 4: Predict the Next Character in a Sequence
    5. Chapter 5: Convolutional Neural Networks for Computer Vision
      1. Activity 5: Making Use of Data Augmentation to Classify correctly Images of Flowers
    6. Chapter 6: Robot Operating System (ROS)
      1. Activity 6: Simulators and Sensor
    7. Chapter 7: Build a Text-Based Dialogue System (Chatbot)
      1. Activity 7: Create a Conversational Agent to Control a Robot
    8. Chapter 8: Object Recognition to Guide a Robot Using CNNs
      1. Activity 8: Multiple Object Detection and Recognition in Video
    9. Chapter 9: Computer Vision for Robotics
      1. Activity 9: A Robotic Security Guard

Product information

  • Title: Artificial Vision and Language Processing for Robotics
  • Author(s): Alvaro Morena Alberola, Gonzalo Molina Gallego, Unai Garay Maestre
  • Release date: April 2019
  • Publisher(s): Packt Publishing
  • ISBN: 9781838552268