Using Transfer Learning to Refine Object Detection Models for Blind and Low Vision Users.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Assistive Technology

Background:

• Object detection models on smartphones, such as YOLOv8, offer potential for assisting people who are blind or low vision (pBLV).
• Existing models may not recognize all objects critical for pBLV navigation.
• The Common Objects in Context (COCO) dataset lacks specific classes relevant to pBLV needs.

Purpose of the Study:

• To compare five transfer learning methods for adapting YOLOv8 object detection models.
• To enhance models with new object classes essential for pBLV navigation.
• To evaluate performance using mean average precision (mAP-50) under varying training conditions.

Main Methods:

• Five transfer learning techniques were applied to YOLOv8s models using a rebalanced COCO dataset.
• Methods included revising all pretrained weights, freezing different numbers of layers (22, 21, 15), and few-shot learning.
• Hyperparameter tuning and application to a larger YOLOv8xl model were also performed.

Main Results:

• Transfer learning methods yielded mAP-50 scores ranging from 0.342 (few-shot learning) to 0.420 (revising all weights).
• The model with 15 frozen layers achieved the highest mAP-50 (0.419), further improved to 0.423 with hyperparameter tuning.
• Applying the best approach to a YOLOv8xl model resulted in an mAP-50 of 0.511.

Conclusions:

• Transfer learning effectively adapts object detection models for pBLV users.
• Even with limited data or computational resources, significant performance gains are achievable.
• These adapted models can enhance navigation tools for people with visual impairments.