Enhancing Emotion Recognition: A Dual-Input Model for Facial Expression Recognition Using Images and Facial Landmarks

Human facial expressions play a fundamental role in nonverbal communication and the conveyance of emotions. Conceptually, facial expressions can be deduced from the arrangement of facial muscles. As a subjective assessment, constructing a database for facial expression recognition becomes a challenge due to the high risk of bias arising from unbalanced or inaccurate data. On the other hand, advances in image processing techniques and deep learning have boosted the accuracy and effectiveness of algorithms for facial expression recognition. In this work, aiming to improve the automatic facial expression recognition, we present the fusion of two neural network architectures. The first one comprises a one-dimensional convolutional neural network (1D), with input characterized by facial landmarks, and a second one, a convolutional neural network based on the DenseNet backbone, with the face image itself as the input. The ADAM optimizer was used during the training of this network. The AffectNet database was employed. The best result obtained was an accuracy of 60.17% in the test subset, for the 7 classes modality. This result is comparable to the best results obtained on the AffectNet dataset.