Text Entry in Virtual Environments using Speech and a Midair Keyboard

Area of Science:

• Human-Computer Interaction
• Virtual Reality Technologies
• Speech Recognition

Background:

• Text input in virtual environments (VE) presents significant usability challenges.
• Lack of effective text input methods hinders immersive VE experiences.
• Existing methods often rely on external input devices, limiting natural interaction.

Purpose of the Study:

• To evaluate the effectiveness of a novel text input system for virtual reality (VR).
• To compare performance metrics of typing versus speech-assisted text entry in VR.
• To assess the impact of hand-tracking and speech recognition on VR text input efficiency.

Main Methods:

• Developed a VR system integrating hand-tracking for mid-air keyboard interaction.
• Implemented a speech recognition component with error correction via word selection.
• Conducted a user study comparing typing-only versus speech-assisted text input in VR.

Main Results:

• Typing-only on the mid-air keyboard yielded 11 words-per-minute (WPM) with 1.2% error rate.
• Speech-assisted input achieved 28 WPM with 0.5% error rate, even with out-of-vocabulary words.
• For in-vocabulary words, speech-assisted input reached 36 WPM with 0.3% error rate.

Conclusions:

• Speech-assisted text input significantly outperforms traditional typing in VR.
• The combined system of hand-tracking, mid-air keyboard, and speech recognition offers an efficient VR text input solution.
• This approach enhances user experience and task completion in virtual environments.