GECO: A Real-Time Computer Vision-Assisted Gesture Controller for Advanced IoT Home System.

Area of Science:

• Human-Computer Interaction
• Internet of Things (IoT)
• Computer Vision

Background:

• Smart home systems require intuitive and accessible control methods.
• Existing interfaces may present barriers for users with specific needs, such as speech impairments.
• Integrating AI for seamless device management is a growing area of research.

Purpose of the Study:

• To introduce GECO, a novel gesture controller for IoT smart homes.
• To enable markerless, real-time control of domestic devices via computer vision.
• To evaluate the system's performance, usability, and inclusivity.

Main Methods:

• Developed a computer vision-assisted gesture controller (GECO) using a markerless MediaPipe interface.
• Integrated binary and analog gestures for device command execution and navigation.
• Implemented on-device processing via a private MQTT network for privacy and low latency.
• Conducted technical evaluations on Android devices and a user experience study.

Main Results:

• Achieved ultra-low latency (<50 ms) for real-time responsiveness.
• Demonstrated high user ratings for intuitiveness (9.5/10) and gesture accuracy (9.2/10).
• Reported significant perceived inclusivity, particularly for users with speech impairments and low technological literacy.

Conclusions:

• GECO provides a lightweight, accessible, and privacy-preserving interaction framework for smart homes.
• The system effectively advances the integration of AI and IoT through intuitive gesture control.
• GECO enhances smart home accessibility and user experience, especially for underserved populations.