Machine Intelligence, Big Data Analytics, and IoT in Image Processing

12Computer Vision-Oriented Gesture Recognition System for Real-Time ISL Prediction

Mukul Joshi, Gayatri Valluri, Jyoti Rawat* and Kriti

School of Computing, DIT University, Dehradun, Uttarakhand, India

Abstract

Sign language is a nonverbal type of correspondence used to transfer information unlike speech, i.e., the verbal type of correspondence that utilizes oral articulations through gestures. This chapter aims to bridge the communication barrier between individuals with vocal and hearing disabilities and non-sign language speakers by creating a recognition model. A system is created using real-time motion analysis where the pre-processing of the dataset is performed and converted to a gray scale. Subsequently, HSV conversion is applied along with skin masking and Canny-edge detection applied for hand tracking and detection. Sign language prediction is performed with the help of a 2-dimensional Convolutional Neural Network (CNN) used mainly in image processing applications with the result being translated and stored into a database such that the signed gesture is recognized and translated to text. The overall accuracy achieved via classification through the 2-D CNN, where 33 labels are detected successfully out of 36 labels is 91%.

Keywords: Visual recognition system, sign language, convolutional neural network, Indian sign language

12.1. Introduction

Sign language is a language incorporating non-verbal correspondence that takes its origin in visual cues and hand gestures. ...