Speech Based Affective Analysis of Patients Embedded in Telemedicine Platforms

Speech is a basic means of human expression, not only due to the combination of words that exits our mouth, but also because of the different way we express these words. Apart from the main objective of speech, which is the communication of information, emotions flow in human speech as various vocal characteristics (prosodic, spectral, tonal). By processing these characteristics, Speech Emotion Recognition aims to analyze and assess the human emotional status to complement medical data captured during telemedicine sessions. Driven by the latest developments in Computer Vision concerning Deep Learning techniques, EfficientNets are exploited to extract features and classify imagery representations of human speech into emotions as a web service along with an interpretation scheme. The developed web service will be consumed during video conferences between medical staff and patients for the near real-time assessment of emotional status of patients during video teleconsultations.