Automatic Detection of Dyspnea in Real Human-Robot Interaction Scenarios

Area of Science:

• Robotics
• Signal Processing
• Human-Robot Interaction

Background:

• A prior respiratory distress estimation technique for telephony was developed.
• Real-world Human-Robot Interaction (HRI) scenarios present unique challenges for audio signal processing due to noise and reverberation.

Purpose of the Study:

• To adapt and evaluate a respiratory distress estimation technique in static and dynamic HRI settings.
• To assess the impact of environmental noise and reverberation on the technique's performance.
• To compare the effectiveness of different beamforming methods and feature combinations.

Main Methods:

• The existing respiratory distress estimation system was re-recorded using a custom robotic platform.
• Telephone training data were augmented with simulated environmental noise and reverberation using room impulse responses (RIRs).
• Performance was evaluated using accuracy and Area Under the Curve (AUC) metrics.
• Delay-and-sum and MVDR beamforming techniques were applied.
• Both time-dependent and time-independent features were analyzed and combined.

Main Results:

• The adapted system achieved accuracy and AUC scores only 0.4% lower than matched simulated data conditions.
• Performance differences between static and dynamic HRI conditions were minimal.
• Beamforming methods (delay-and-sum, MVDR) improved accuracy by 8% and AUC by 2% on average.
• Combining time-dependent and time-independent features yielded the best joint accuracy and AUC.

Conclusions:

• The respiratory distress estimation technique is robust and adaptable to real-world static and dynamic HRI environments.
• Environmental modeling and beamforming techniques significantly enhance performance.
• A combination of diverse features offers the most effective approach for accurate respiratory distress estimation in HRI.