Inertial Sensor-Based Touch and Shake Metaphor for Expressive Control of 3D Virtual Avatars

Area of Science:

• Human-Computer Interaction
• Virtual Reality
• Robotics

Background:

• Controlling virtual avatars requires intuitive and expressive input methods.
• Existing methods often lack the nuance for detailed character animation.
• Inertial sensors offer a promising avenue for gesture-based control.

Purpose of the Study:

• To develop an inertial sensor-based system for expressive control of 3D virtual avatars.
• To enable real-time tracking and recognition of dynamic hand gestures.
• To synthesize novel avatar motions beyond a pre-existing database.

Main Methods:

• Utilized a six degrees-of-freedom wireless inertial motion sensor.
• Implemented a sensor fusion algorithm with magnetic, angular rate, and gravity sensors.
• Employed a quaternion-based complementary filter for noise reduction and dynamic time-warping for gesture recognition.

Main Results:

• Successfully tracked user hand motions in 3D space.
• Achieved efficient recognition of dynamic hand gestures with real-time segmentation.
• Demonstrated gesture expressivity through a flexible gesture mapping interface.
• Synthesized stylistic variations and novel motions for a 3D virtual avatar.

Conclusions:

• The proposed inertial sensor system provides an intuitive and expressive method for controlling 3D virtual avatars.
• Dynamic hand gesture recognition enables continuous and nuanced avatar interaction.
• This approach expands the expressive capabilities of virtual avatars beyond motion capture databases.