Traffic Light Recognition Assistant for Color Vision Deficiency Using YOLO with Multilingual Audio Feedback

Area of Science:

• Intelligent Transportation Systems (ITSs)
• Computer Vision
• Assistive Technologies

Background:

• Drivers with color vision deficiency (CVD) face significant challenges in recognizing traffic light colors, increasing driving risks.
• Existing traffic signal recognition systems often rely on color perception, posing limitations for individuals with CVD.
• The need for inclusive Intelligent Transportation Systems (ITSs) that cater to diverse user needs, including those with visual impairments, is growing.

Purpose of the Study:

• To develop an innovative traffic light recognition system for drivers with CVD.
• To provide offline, multilingual audio feedback (Indonesian, Mandarin, English) for enhanced user experience.
• To reduce reliance on color-based perception in traffic signal recognition for assistive ITS applications.

Main Methods:

• Implementation of a spatial-position inference framework using a YOLOv12 model with full-frame traffic light annotation.
• Traffic light state recognition based on the relative positions of active lights, minimizing dependence on color information.
• System evaluation using a comprehensive dataset encompassing diverse driving conditions (day/night, various weather, traffic densities).

Main Results:

• The system achieved an average detection confidence of approximately 0.73, with a maximum confidence of 0.95.
• Demonstrated low processing latency of 0.214 seconds on a CPU-only configuration, indicating real-time feasibility.
• Successfully recognized traffic light states using spatial positioning rather than solely color.

Conclusions:

• The developed traffic light recognition system offers a viable assistive tool for drivers with CVD.
• The system enhances driving safety by providing an intelligent aid within ITS frameworks.
• This approach contributes to more inclusive and accessible transportation for individuals with color vision deficiency.