Slot: a research platform for investigating multimodal communication

Area of Science:

• Human-Computer Interaction
• Cognitive Science
• Communication Studies

Background:

• Multimodal communication research requires flexible experimental platforms.
• Understanding the interplay between speech and non-verbal cues like pen gestures is crucial.

Purpose of the Study:

• To introduce the Spatial Logistics Task (SLOT) platform for multimodal communication research.
• To detail the SLOT communication task, software, and hardware for experimental setup and data recording.
• To demonstrate the platform's utility in studying human-human and human-computer interactions.

Main Methods:

• Development of the SLOT platform, including a specific communication task, software, and hardware.
• Conducting experiments using the SLOT platform, such as a negotiation task with a one-way mirror.
• Performing exploratory studies on automatic recognition of spontaneous pen gestures.

Main Results:

• The SLOT platform provides high flexibility in manipulating communication contexts.
• Early experiments highlight the platform's effectiveness in capturing multimodal behavior.
• Demonstrated the platform's utility in analyzing the relationship between pen gestures and speech.

Conclusions:

• The SLOT platform is a valuable tool for multimodal communication research.
• The platform supports investigations into both human-human and human-computer interaction dynamics.
• The findings underscore the importance of integrated approaches to studying communication.