Saliency-driven real-time video-to-tactile translation.

Area of Science:

• Human-Computer Interaction
• Multimodal Systems
• Haptic Technology

Background:

• Visuotactile stimuli enhance immersive experiences but traditionally require manual authoring.
• Content artists have been essential for creating synchronized tactile feedback for visual content.
• Existing methods for generating tactile cues are labor-intensive and time-consuming.

Purpose of the Study:

• To present a fully automated framework for generating real-time tactile cues from streaming images.
• To enable synchronized visuotactile stimuli for enhanced user experiences.
• To evaluate the effectiveness of automated tactile cue generation compared to manual methods.

Main Methods:

• Analyzing spatiotemporal features of video images based on visual saliency.
• Mapping visual features to tactile cues rendered on chair-installed tactors.
• Conducting user experiments to evaluate visuotactile rendering and authoring methods.

Main Results:

• Visuotactile rendering significantly improved movie watching experiences, making them more interesting, immersive, and understandable compared to visual-only rendering.
• The automated authoring approach, when carefully applied, can produce tactile effects of quality comparable to manual authoring.
• User experiments confirmed the effectiveness of the proposed automated framework.

Conclusions:

• Automated generation of synchronized tactile feedback is feasible and effective.
• The proposed framework offers a viable alternative to manual authoring for visuotactile experiences.
• This technology has the potential to democratize the creation of immersive multimodal content.