YOLO Series for Human Hand Action Detection and Classification from Egocentric Videos.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Hand detection and classification are crucial for 3D hand pose estimation and activity recognition.
• Egocentric vision (EV) datasets present unique challenges for hand data analysis.
• The evolution of the You Only Live Once (YOLO) network necessitates a comparative study of its family for hand-related tasks.

Purpose of the Study:

• To systematically review YOLO-family network architectures (v1-v7), their pros, and cons.
• To curate ground-truth data for hand detection and classification on EV datasets (FPHAB, HOI4D, RehabHand).
• To evaluate and compare the efficiency of YOLO-family networks for hand detection and classification on EV datasets.

Main Methods:

• Systematic literature review of YOLO versions 1 through 7.
• Dataset preparation and annotation for FPHAB, HOI4D, and RehabHand.
• Fine-tuning and evaluation of YOLO-family models on prepared EV datasets.

Main Results:

• YOLOv7 and its variants achieved the highest accuracy in hand detection and classification across all tested EV datasets.
• YOLOv7-w6 achieved 97% precision on FPHAB, 95% on HOI4D, and over 95% on RehabHand (with a threshold of 0.5).
• YOLOv7-w6 processed at 60 fps (1280x1280), while YOLOv7 achieved 133 fps (640x640), indicating strong performance-speed trade-offs.

Conclusions:

• YOLOv7-w6 is highly effective for hand detection and classification in egocentric vision tasks.
• The YOLOv7 architecture offers a robust solution for real-time 3D hand pose estimation and activity recognition.
• Comparative analysis validates YOLOv7's superiority for hand analysis in EV datasets.