A hand sign recognition based signal system for mute people using machine learning

Area of Science:

• Computer Science
• Human-Computer Interaction
• Assistive Technology

Background:

• Effective communication is vital, but individuals with speech impairments face significant barriers.
• Existing sign language recognition systems are often computationally intensive, limiting real-world application on modest hardware.
• Lack of sign language proficiency in the general public exacerbates communication challenges for the mute community.

Purpose of the Study:

• To develop a lightweight and efficient hand gesture recognition framework for real-time communication.
• To enable intuitive communication for individuals with speech deficiencies using accessible technology.
• To bridge the communication gap between mute individuals and the general public.

Main Methods:

• Integration of MediaPipe hand landmark detection with dual classifiers for static and dynamic gesture recognition.
• Development of a lightweight framework suitable for deployment on low-resource hardware.
• Testing with a balanced dataset of 3500 samples across seven representative gestures (A, B, C, D, Open, Close, OK).

Main Results:

• Achieved 94.1% accuracy on a partitioned test set.
• Sustained real-time performance at 30 frames per second (FPS) on CPU-only deployment.
• Demonstrated robustness across varied lighting conditions.

Conclusions:

• The proposed framework offers a high-performing balance of accuracy, efficiency, and accessibility for assistive communication.
• This approach provides a viable, low-resource method for real-time hand gesture recognition.
• The system enhances communication accessibility for individuals with speech impairments.