Data Engineering and Data Science

9Unobtrusive Engagement Detection through Semantic Pose Estimation and Lightweight ResNet for an Online Class Environment

Michael Moses Thiruthuvanathan¹*, Balachandran Krishnan² and Madhavi Rangaswamy²

¹ Department of CSE, School of Engineering and Technology, CHRIST (Deemed to be University), Bengaluru, Karnataka, India

² Department of Psychology, School of Social Sciences, CHRIST (Deemed to be University), Bangalore, Karnataka, India

Abstract

Analysing student engagement in a class through unobtrusive methods enhances the learning and teaching experience. During these pandemic times, where the classes are conducted online, it is imperative to efficiently estimate the engagement levels of individual students. Helping teachers to annotate and understand the significant learning rate of the students is critical and vital. To facilitate the analysis of estimating the engagement levels among students, this paper proposes a dual channel model to precisely detect the attention level of individual students in a classroom. Considering the possible inaccuracy of emotion recognition, a dual channel is configured with a Lightweight ResNet model for macro-level attention estimation and a 3d pose estimation using Euler angles for Pitch, yaw and roll that is trained, validated and tested on the Daisee database. The Emotional detection extracts the context of Engaged, frustrated, confused and disgust as higher levels of classroom attention cognition while the facial pose coordinates provide ...