ThunderPunch: A bare-hand, gesture-based, large interactive display interface with upper-body-part detection in a top...

Area of Science:

• Human-Computer Interaction
• Computer Vision
• Interactive Systems

Background:

• Traditional large-screen interaction often requires physical contact or limited gesture recognition capabilities.
• Existing systems face challenges with multi-user participation and natural interaction methods.
• Camera placement in conventional setups can obstruct the display area.

Purpose of the Study:

• To develop a novel bare-hand gesture interface for simultaneous multi-user interaction on large screens.
• To introduce a new hardware system, ThunderPunch, with a large hybrid display and ceiling-mounted cameras.
• To propose real-time algorithms for detecting multiple body poses and recognizing specific gestures from top-view depth images.

Main Methods:

• Development of the ThunderPunch hardware system featuring a large hybrid display and ceiling-mounted cameras.
• Implementation of real-time algorithms for multi-body pose detection from top-view depth images.
• Gesture recognition algorithms specifically designed for punching and touching interactions.

Main Results:

• The proposed algorithms successfully detect multiple body poses and recognize punching and touching gestures.
• A pointing and touching test demonstrated the usability and superior performance of the proposed algorithm compared to others.
• A game was developed to showcase the capabilities of the ThunderPunch system and the gesture interface.

Conclusions:

• The developed bare-hand gesture interface and ThunderPunch system enable intuitive, simultaneous multi-user interaction on large screens.
• The proposed real-time algorithms provide effective gesture recognition from top-view depth images, overcoming conventional limitations.
• The system offers a promising new direction for interactive large-screen experiences, validated through user testing and application development.