Automatic Speech Recognition and Translation for Low Resource Languages

8A New Robust Deep Learning-Based Automatic Speech Recognition and Machine Transition Model for Tamil and Gujarati

Monesh Kumar M. K.¹, Valliammai V.¹, Geraldine Bessie Amali D.¹* and Mathew M. Noel²

¹School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India

²School of Electrical Engineering, Vellore Institute of Technology, Vellore, India

Abstract

This chapter proposes a new robust deep learning-based automatic speech recognition (ASR) and machine translation system for Tamil to Gujarati and vice versa. Tamil and Gujarati are both regional languages in India. Tamil is the oldest language in the world while Gujarati has around 46 million native speakers. Tourism and trade can flourish between the two states if the language barrier is broken. Recent extensive research in the field of natural language processing has led to the creation of various interfaces that can be used in solving the problem. The proposed system achieves the task by performing the following sequence of steps: listen, attend, and spell (LAS) model for ASR followed by neural machine translation (NMT) such as multilingual transformer (mT5) and then neural text-tospeech (TTS) system. The LAS model takes as input a sequence of audio features extracted from the raw audio signal, and it uses an attention mechanism to focus on different parts of the audio sequence while decoding a sequence of phonemes or words representing the transcription. Deep learning techniques are employed ...