Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Relationship among Sleep Disturbance, Stress, and Suicidal Ideation in Clinical High Risk for Psychosis.

Schizophrenia bulletin open·2026

Same author

Premorbid adjustment problems, negative symptoms, and cognitive impairment in a large international sample at clinical high risk for psychosis: Findings from the Accelerating Medicines Partnership-Schizophrenia.

Schizophrenia bulletin·2026

Same author

Common Electrophysiology Biomarkers Collected at Home Robustly Track Depression Recovery With Deep Brain Stimulation.

medRxiv : the preprint server for health sciences·2026

Same author

Alzheimer's Disease Brain Phenotypes are Age-dependent.

bioRxiv : the preprint server for biology·2026

Same author

The Utility of Negative Symptoms in Predicting Transition to Psychosis Among Individuals at Clinical High Risk.

Schizophrenia bulletin·2026

Same author

Machine Learning-Based Audiovisual Phenotyping for Measuring Communication, Shared Decision-Making, and Trust.

JMIR formative research·2026

Search research articles

Related Experiment Video

Updated: Oct 5, 2025

Asthma Detection Research Based on Voice Signal Processing and Machine Learning

Asthma Detection Research Based on Voice Signal Processing and Machine Learning

Published on: July 22, 2025

Acoustic and Facial Features From Clinical Interviews for Machine Learning-Based Psychiatric Diagnosis: Algorithm

Michael L Birnbaum^1,2,3, Avner Abrami⁴, Stephen Heisig⁵

¹Department of Psychiatry, The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, United States.

JMIR Mental Health

|January 24, 2022

Summary

Objective psychiatric assessments are now possible using machine learning analysis of facial and vocal patterns. This technology can aid in diagnosing schizophrenia spectrum and bipolar disorders, improving patient care.

Keywords:

audiovisual audiovisual patterns bipolar disorder diagnostic prediction facial analysis machine learning psychiatry schizophrenia schizophrenia spectrum disorders spectrum disorders speech speech analysis symptom prediction

More Related Videos

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

Olfactory Neurons Obtained through Nasal Biopsy Combined with Laser-Capture Microdissection: A Potential Approach to Study Treatment Response in Mental Disorders

Olfactory Neurons Obtained through Nasal Biopsy Combined with Laser-Capture Microdissection: A Potential Approach to Study Treatment Response in Mental Disorders

Published on: December 4, 2014

Related Experiment Videos

Last Updated: Oct 5, 2025

Asthma Detection Research Based on Voice Signal Processing and Machine Learning

Asthma Detection Research Based on Voice Signal Processing and Machine Learning

Published on: July 22, 2025

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

Olfactory Neurons Obtained through Nasal Biopsy Combined with Laser-Capture Microdissection: A Potential Approach to Study Treatment Response in Mental Disorders

Olfactory Neurons Obtained through Nasal Biopsy Combined with Laser-Capture Microdissection: A Potential Approach to Study Treatment Response in Mental Disorders

Published on: December 4, 2014

Area of Science:

Psychiatry
Machine Learning
Digital Health

Background:

Psychiatry relies heavily on subjective assessments, impacting care quality.
Objective data is crucial for effective behavioral health interventions.

Purpose of the Study:

Investigate extracting psychiatric diagnoses and symptoms from audiovisual data.
Utilize machine learning to analyze patterns in patient interviews.

Main Methods:

Collected audiovisual data from 89 participants with schizophrenia spectrum disorders, bipolar disorder, and healthy controls.
Developed machine learning models using acoustic and facial movement features.
Assessed model performance using area under the receiver operating characteristic curve (AUROC).

Main Results:

Differentiated schizophrenia spectrum disorders from bipolar disorder (AUROC 0.73) using combined features.
Identified key facial and vocal features for diagnosis in men and women.
Successfully inferred psychiatric symptoms like blunted affect (AUROC 0.81) and avolition (AUROC 0.72).

Conclusions:

Demonstrates advancement in using digital data for psychiatric assessment.
Supports development of innovative clinical tools using acoustic and facial analysis.