Detecting paralinguistic events in audio stream using context in features and probabilistic decisions.

Area of Science:

• Speech processing and human-computer interaction
• Computational linguistics and affective computing
• Machine learning for signal processing

Background:

• Non-verbal communication, including vocal cues like laughter and fillers, is crucial for conversational flow, emotion expression, and personality marking.
• Automatic detection of these non-verbal vocal events is challenging but valuable for understanding human interaction.
• Previous work established a baseline system in the Interspeech 2013 Social Signals Sub-challenge.

Purpose of the Study:

• To develop and enhance an automatic system for detecting non-verbal vocal events, specifically laughter and fillers.
• To investigate the incorporation of local context at feature and decision levels for improved detection accuracy.
• To analyze feature sensitivity and its impact on system design for non-verbal vocal event detection.

Main Methods:

• Extension of a previous winning system from Interspeech 2013 for frame-wise event detection.
• Incorporation of local context at two levels: raw frame-wise features and output decisions.
• Application of heuristic rules to refine frame-based predictions and reduce errors.

Main Results:

• Achieved an Area Under the ROC curve of 95.3% for laughter detection.
• Achieved an Area Under the ROC curve of 90.4% for filler detection on the Interspeech 2013 test set.
• Feature sensitivity analysis indicated that features with higher discriminability positively impact system performance.

Conclusions:

• The developed system effectively detects laughter and fillers with high accuracy, outperforming previous benchmarks.
• Incorporating local context and heuristic post-processing significantly enhances the robustness of non-verbal vocal event detection.
• Understanding feature importance is key for designing more effective and sensitive automatic speech analysis systems.