Related Experiment Video
Updated: Jul 1, 2026
Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury
Published on: May 15, 2016
A client-server based recognition system: Non-contact single/multiple emotional and behavioral state assessment
Xianxun Zhu1, Zhaozhao Liu1, Erik Cambria2
1School of Communication and Information Engineering, Shanghai University, 200444, Shanghai, China.
Background And Objectives:
In the current global health landscape, there is an increasing demand for rapid and accurate assessment of mental states. Traditional assessment methods typically rely on face-to-face interactions, which are not only time-consuming but also highly subjective. Addressing this issue, this study aims to develop a client-server-based, non-contact multimodal emotion and behavior recognition system to enhance the efficiency and accuracy of mental state assessments.
Methods:
This study designed and implemented a multimodal assessment system integrating voice, text, facial expressions, and body movements. Utilizing a client-server architecture, the system optimizes diagnostic efficiency and decision-making accuracy through an intuitive visual interface. The system's effectiveness was validated and tested in actual hospital settings.
Results:
The system demonstrated exceptional performance in multimodal emotion and behavior recognition, achieving a voice recognition accuracy of 92.01%, facial expression recognition accuracy of 91.3%, and an overall multimodal assessment accuracy of 77.9%. Moreover, it reached a behavior analysis accuracy of 94.5%.
Conclusions:
The multimodal assessment system developed in this study significantly enhances the accuracy and efficiency of mental state assessments, meeting the needs of clinicians for precise and rapid diagnostics in real-world settings.
