Automatic Behavior Assessment from Uncontrolled Everyday Audio Recordings by Deep Learning.

Area of Science:

• Behavioral Science
• Machine Learning
• Audio Signal Processing

Background:

• Manual categorization of behavior from sensory data is costly and subjective, often requiring multiple domain experts.
• Automating behavior analysis can reduce costs and improve objectivity.
• Uncontrolled everyday audio recordings offer a rich source of behavioral data.

Purpose of the Study:

• To investigate the feasibility of automatically coding social behavior and environments using deep learning on uncontrolled everyday audio recordings.
• To assess the effectiveness of transfer learning for audio-based behavior classification.
• To identify challenges in automatic behavior assessment from ambient audio data.

Main Methods:

• Collected daily living audio recordings from healthy young and older adults using wearable devices.
• Applied a transfer learning approach using a pretrained neural network followed by fine-tuning for audio classification.
• Utilized deep learning models to analyze audio data for behavior and environment coding.

Main Results:

• Deep learning models demonstrated the potential to automatically classify certain aspects of social behavior and environments.
• Transfer learning proved effective for adapting pretrained models to the task of audio-based behavior analysis.
• Ambient noise in uncontrolled recordings significantly challenged the accuracy of automatic behavior assessment.

Conclusions:

• Automatic coding of social behavior and environments from everyday audio is feasible with deep learning, but challenges remain.
• Ambient noise and data sparsity are critical limitations for reliable automatic behavior assessment in real-world audio.
• Further research is needed to enhance robustness against noise and data limitations for practical applications.