Deep Learning-Based Real-Time Facial Expression Recognition Systems for Assessing Emotional and Cognitive Engagement in Special Education Classrooms

Abstract

Deep learning-based real-time facial expression recognition systems offer a supportive approach for assessing emotional and cognitive engagement indicators in special education classrooms. This study developed a facial emotion recognition framework using an autism-focused facial emotion dataset to classify six emotional categories: anger, fear, joy, natural expression, sadness, and surprise. The dataset contained 1,425 labeled images, including 1,199 training images and 226 testing images, and was evaluated for structure, image quality, preprocessing compatibility, class distribution, and baseline classification performance. All images were readable, no corrupt files were detected, and the class distribution was balanced across the six emotion categories. Images were prepared through resizing, normalization, categorical label encoding, and augmentation planning. A transfer-learning-based convolutional neural network framework was proposed for emotion classification, while baseline analysis using handcrafted image features produced limited performance, with 24.8% test accuracy and 25.4% macro-F1 score. Predicted emotions were mapped into emotional engagement and cognitive engagement proxy indicators, where joy and natural expression represented positive or stable engagement, surprise indicated heightened attention, and anger, fear, and sadness indicated at-risk engagement. The findings support the feasibility of a classroom-oriented facial expression recognition framework, although direct classroom validation and cognitive engagement labels remain necessary for future development.