Building a Multi-Sensor Lab for Interactive Communication Research: Challenges, Workarounds, and Lessons Learned

Area of Science:

• Human-Computer Interaction
• Communication Sciences
• Biomedical Engineering

Background:

• Designing multi-sensor experiments for interactive communication is complex and costly.
• Implementation challenges and decision-making rationales are often poorly documented.
• A need exists for detailed guidance on developing multi-sensor labs for communication research.

Purpose of the Study:

• To demystify the process of designing and implementing custom multi-sensor labs.
• To detail the development of a specific multi-sensor lab for monitoring interactive communication.
• To share challenges and solutions for building robust multi-sensor experimental setups.

Main Methods:

• Developed a custom multi-sensor lab to monitor behavioral and physiological responses during natural conversation.
• Integrated various sensors including headset microphones, eye-tracking glasses, motion capture, physiological amplifiers, a smartwatch, and video cameras.
• Implemented a central soundcard for trigger pulses and developed bespoke processing pipelines for data synchronization due to high bandwidth demands and varying sampling rates.

Main Results:

• Successfully collected multi-sensor data from two adults conversing in a controlled environment.
• Identified significant synchronization challenges due to diverse data streams and sampling rates.
• Developed and implemented redundant synchronization mechanisms and custom processing pipelines to ensure data integrity.

Conclusions:

• The developed multi-sensor lab provides a comprehensive system for studying interactive communication.
• Sharing implementation challenges and solutions can aid researchers in building similar setups.
• Informed decision-making and anticipation of common pain points are crucial for successful multi-sensor lab development.