Multi-Label Emotion Detection for Mental Health Monitoring Using Deep CNN and Visual Attention

Sari

Mental health is a crucial aspect of modern human life that requires continuous monitoring. This research aims to develop a multi-label classification system to automatically detect various human emotional expressions through facial images, serving as a supportive approach for AI-based mental health monitoring. The proposed system leverages a Convolutional Neural Network (CNN) architecture integrated with a visual attention mechanism using the Convolutional Block Attention Module (CBAM). The AffectNet dataset is used as the primary data source, providing multi-label annotations for various emotional states. The model is designed using sigmoid activation and binary cross-entropy loss to handle multiple emotions simultaneously in a single image. Evaluation is conducted using the confusion matrix and metrics such as Precision, Recall, and F1-score. Experimental results demonstrate that the model achieves a Mean Average Precision (mAP) of 89.7%, indicating good performance in multi-label emotion classification. Specifically, the model achieves an F1-score of 100% for the emotions Happy, Fear, Surprise, Disgust, and Neutral, but faces challenges in distinguishing Sad (F1-score 67%) and Angry (F1-score 80%) expressions from others. Incorporating the attention mechanism proves beneficial in enhancing the overall performance of the model. This study contributes to the development of adaptive emotion recognition technologies, potentially applicable in real-time, non-invasive psychological monitoring systems.