Related Experiment Video
Updated: Aug 5, 2026
05:10
An Automated Method for Assessing Visual Acuity in Infants and Toddlers Using an Eye-Tracking System
Published on: March 17, 2023
Thrivaad: A Multilingual, Predictive Eye-Sign-Based AAC System Powered by Optimized Deep Learning
Rajesh Kannan Megalingam1, Sakthiprasad Kuttankulangara Manoharan1, Dhilna Cheriyan Manjooran1
1Humanitarian Technology (HuT) Labs, Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kollam 690525, Kerrla, India.
Sensors (Basel, Switzerland)
|July 28, 2026
Summary
Thrivaad enhances Augmentative and Alternative Communication (AAC) for individuals with speech impairments. This eye-gaze system offers multilingual support and text prediction, improving communication accuracy and adaptability in diverse conditions.
Area of Science:
- Assistive Technology
- Human-Computer Interaction
- Biomedical Engineering
Background:
- Speech impairments affect 1.5% of the global population, primarily due to cerebral palsy and ALS.
- Augmentative and Alternative Communication (AAC) systems are crucial for communication, with electronic methods like Brain-Computer Interaction (BCI) and eye-gaze systems showing promise.
- Existing electronic AAC systems face limitations in adaptability to changing conditions (e.g., lighting, user fatigue) and often lack multilingual support.
Purpose of the Study:
- To develop Thrivaad, an advanced eye-gaze-based AAC system.
- To enhance communication robustness and user experience for individuals with speech impairments.
- To provide multilingual support and text prediction capabilities in an eye-tracking system.
Main Methods:
- Utilized eye movements captured via webcam as input.
- Employed an Optuna-optimized YOLOv5 model for accurate eye direction detection.
- Integrated deep learning for improved eye movement capture under varying lighting conditions.
- Implemented text prediction to reduce required eye gestures.
Main Results:
- Achieved accurate eye direction detection using the optimized YOLOv5 model.
- Demonstrated multilingual communication support in English, Malayalam, and Hindi.
- Showcased improved adaptability to changing environmental lighting conditions.
- Validated the text-prediction feature's efficiency in message composition.
Conclusions:
- Thrivaad offers a robust, adaptable, and multilingual eye-gaze-based AAC solution.
- The system effectively addresses limitations of previous AAC technologies, particularly in variable lighting and user fatigue.
- Thrivaad has the potential to significantly improve communication for individuals with speech impairments.