Eliciting Multimodal and Collaborative Interactions for Data Exploration on Large Vertical Displays.

Area of Science:

• Human-Computer Interaction
• Collaborative Systems
• Interactive Data Visualization

Background:

• Large vertical displays enable rich visual data exploration and multi-user interaction.
• Integrating multiple interaction modalities (touch, speech, pen, mid-air gestures) presents significant design challenges.

Purpose of the Study:

• To investigate user preferences for combining interaction modalities in collaborative settings on large vertical displays.
• To understand how users associate specific tasks with interaction modalities and collaboration styles.

Main Methods:

• An elicitation study involving 20 participants was conducted.
• Participants generated 1015 interaction proposals combining touch, speech, pen, and mid-air gestures.
• User preferences for different modalities across various tasks were analyzed.

Main Results:

• Speech interaction was preferred for 10 out of 15 low-level tasks.
• Direct manipulation was favored for simple tasks like showing tooltips or selections.
• Contrary to prior research, participants predominantly favored unimodal and personal interactions over multimodal ones.
• Collaborative synonyms were identified, with pairs collaborating either unimodally and simultaneously or multimodally and sequentially.

Conclusions:

• End-users associate specific visual exploration tasks with particular interaction modalities.
• Collaboration on large vertical displays can occur effectively through unimodal and simultaneous interactions, challenging previous assumptions.
• Insights are provided for designing collaborative systems that accommodate diverse user preferences and interaction distances.