EgoVision a YOLO-ViT hybrid for robust egocentric object recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Robotics

Background:

• Egocentric vision, or first-person perspective, is crucial for assistive technologies, augmented reality, and human-computer interaction.
• Object recognition in egocentric vision faces challenges like occlusion, motion blur, and viewpoint changes.
• Existing methods struggle with the computational demands of real-time egocentric object recognition.

Purpose of the Study:

• To introduce EgoVision, a novel hybrid deep learning framework for object classification in static egocentric frames.
• To fuse the spatial precision of YOLOv8 with the global contextual reasoning of Vision Transformers (ViT).
• To enable real-time object recognition for applications in robotics and augmented reality using the HOI4D dataset.

Main Methods:

• Developed EgoVision, a lightweight hybrid deep learning framework combining YOLOv8 and Vision Transformers (ViT).
• Employed a key-frame extraction strategy and a feature pyramid network to efficiently process multiscale spatial-temporal features.
• Utilized the HOI4D dataset for training and evaluating static object recognition in egocentric frames.

Main Results:

• EgoVision achieved up to 99% accuracy on complex object classes like 'Kettle' and 'Chair'.
• The framework demonstrated superior performance across multiple metrics compared to existing models.
• EgoVision maintained high efficiency, suitable for deployment on wearable and edge devices.

Conclusions:

• EgoVision represents a significant advancement in egocentric object recognition, offering high accuracy and efficiency.
• The hybrid architecture effectively addresses the challenges of first-person perspective object classification.
• EgoVision provides a robust foundation for next-generation egocentric AI systems in real-time applications.