Low-Cost Human-Machine Interface for Computer Control with Facial Landmark Detection and Voice Commands

Area of Science:

• Computer Science
• Human-Computer Interaction
• Rehabilitation Engineering

Background:

• Modern society necessitates extensive computer interaction, posing challenges for individuals with upper limb disabilities.
• Existing assistive technologies may not fully address the need for seamless computer control for all users.

Purpose of the Study:

• To develop and evaluate an innovative interface, EMKEY (Emulating Mouse and Keyboard functions), for computer control using artificial vision and voice recognition.
• To assess the usability and effectiveness of EMKEY for individuals with and without physical disabilities.

Main Methods:

• Developed EMKEY interface integrating head movement for pointer control and voice recognition for command execution and speech-to-text.
• Conducted two studies: one with 30 unimpaired participants to analyze usability correlations, and a second with four participants with motor disabilities to assess real-world application.

Main Results:

• The first study revealed significant correlations between EMKEY usability and adaptability, execution time, and participant age.
• The second study indicated EMKEY was most effective for a participant with cerebral palsy, followed by those with upper limb paralysis, spina bifida, and muscular dystrophy.

Conclusions:

• EMKEY is a user-friendly, practical, and accurate interface adaptable to various computer systems.
• The developed interface demonstrates significant potential as an assistive technology for individuals with motor impairments, improving their computer interaction capabilities.