Related Experiment Video
Updated: May 21, 2026
Resting-State Connectivity and Neuroimaging of Prefrontal Cortex Activity During a Block-Design Yoga Asana Practice Using fNIRS
Published on: June 24, 2025
Enhancing Yoga Pose Recognition through a Multimodal Hybrid Approach with Attention Mechanism and Late Fusion on Yoga
Jayasurya Marasani1, Dhanvanth Reddy Yerramreddy1, S Don2
1Department of Computer Science and Engineering, Amrita School of Computing, Amrita Vishwa Vidyapeetham, Kollam, Kerala, India.
Context:
The project focuses on yoga pose recognition, a critical task in computer vision and AI, to enhance physical and mental well-being by improving posture analysis. It addresses challenges such as the lack of benchmark datasets and limitations in both landmark-augmented and landmark-free methods.
Aims:
1. Developed a hybrid model combining landmark-based and landmark-free techniques for accurate yoga pose recognition 2. Introduced YFA-30, a comprehensive dataset with 30 yoga poses to benchmark pose recognition models 3. Conducted a comparative analysis of existing models with the proposed hybrid model.
Settings And Design:
The project operates within the domain of computer vision and artificial intelligence, specifically focusing on yoga pose recognition. It involves a controlled computational environment for training, validating, and testing deep learning models using datasets of yoga poses. The settings emphasize creating a system capable of real-time pose recognition and correction for practical applications in fitness, health, and education.
Methods:
• Hybrid Model: Combines MoveNet (landmark detection) and ResNet50 (image feature extraction) using a scaled dot-product attention mechanism and late fusion • Statistical Evaluation: Analyzes accuracy, loss, and performance metrics across multiple datasets (YFA-30, YFA-20, YFA-10, Kaggle, and Google CG) • Comparative Study: Benchmarks against models such as Mediapipe, YOLO-V8x, and Xception • Visualization: Uses Grad-CAM heatmaps for feature extraction insights.
Material:
Datasets: • YFA-30: Newly introduced dataset with 30 yoga poses with approximately 8000 images • Kaggle Dataset: Small dataset with 5 yoga poses • Google CG Dataset: Synthetic images of 5 yoga poses. Models: • Landmark-based: MoveNet, Mediapipe, and YOLO-V8x • Landmark-free: ResNet50, VGG16, EfficientNetB0, and Xception.
Statistical Analysis Used:
The statistical analyses used in your yoga pose recognition project include: 1. Performance Metrics: Accuracy, loss (training, validation, and test), and loss curves across datasets (YFA-30, YFA-20, YFA-10, Kaggle, and Google CG) to evaluate model efficiency 2. Comparative Analysis: Benchmarking the hybrid model (MoveNet + ResNet50) against other models such as Mediapipe, YOLO-V8x-pose, and landmark-free models (e.g., VGG16 and Xception) 3. Cross-Validation: Testing on YFA dataset subsets to ensure robustness across data variability 4. Attention Mechanism: Scaled dot-product attention and late fusion to integrate and prioritize relevant features and landmarks 5. Visual Analysis: Heatmaps (Grad-CAM) to interpret keypoint detection and feature extraction 6. Dataset Testing: Evaluation on synthetic and real-world datasets to ensure adaptability. These analyses validate the hybrid model's performance and robustness.
