Voice and Touch Based Error-tolerant Multimodal Text Editing and Correction for Smartphones

Area of Science:

• Human-Computer Interaction
• Mobile Computing
• Natural Language Processing

Background:

• Smartphone text editing (cut, copy, paste, error correction) is often inefficient and challenging.
• Existing methods like touch-only or voice control require precise cursor control or text selection, leading to user frustration.

Purpose of the Study:

• To introduce VT, a novel voice and touch-based multi-modal method for efficient text editing and correction on smartphones.
• To leverage language context to accurately infer user editing intentions, handling input ambiguities and noise.

Main Methods:

• VT combines touch gestures (gliding, tapping) with voice commands for text manipulation and correction.
• It utilizes language models and phrase similarity to interpret user intent, improving accuracy over traditional methods.
• Users can edit text by gliding over fragments and dictating commands or by tapping and speaking corrections.

Main Results:

• VT significantly enhances the efficiency of text editing and correction on smartphones.
• User studies demonstrated VT reduced text editing time by 30.80% and text correction time by 29.97% compared to touch-only methods.
• VT outperformed iOS Voice Control, reducing editing time by 30.81% and correction time by 47.96%.

Conclusions:

• VT offers a more efficient and user-friendly approach to smartphone text editing and correction.
• The multi-modal strategy effectively addresses the limitations of existing text manipulation techniques on mobile devices.