Related Experiment Videos
C-TransDance: A Style-Conditioned Gated CNN-Transformer for Real-Time Multi-Style Dance Quality Assessment Using
Shan Zhang1, Yantong Liu2, Seong-Yoon Shin2
1Department of Dance, Sookmyung Women's University, Seoul 04310, Republic of Korea.
Abstract:
Vision-based human motion assessment requires a model to capture both temporal execution patterns and context-dependent quality criteria. This problem is particularly challenging in multi-style dance, where similar kinematic patterns may receive different quality evaluations under different stylistic conventions. We propose C-TransDance, a style-conditioned gated CNN-Transformer for real-time multi-style dance quality assessment using pose-based visual sensing. The system extracts 33 body landmarks from a single front-facing camera with MediaPipe BlazePose GHUM and represents each motion sequence using normalized joint coordinates, confidence, velocity, and acceleration. A convolutional branch for local spatio-temporal execution patterns and a Transformer branch for long-range temporal dependencies are combined through learnable gated fusion and modulated by a learned dance style embedding with an explicit feature-style interaction. Multi-task heads jointly predict continuous quality scores, quality classes, and structured feedback categories, and temporal attention and joint-level sensitivity are coupled with constrained feedback templates to expose inspectable movement evidence. The framework was evaluated on 1152 clips from 96 participants across four dance styles, annotated independently by three qualified instructors, using participant-independent splits and five random seeds. On the held-out test set (173 clips, 15 unseen participants), C-TransDance achieved an MAE of 4.86 ± 0.16, an RMSE of 6.38 ± 0.22, an accuracy of 82.1% ± 1.1%, and a macro-F1 of 0.822 ± 0.011 compared with an MAE of 5.63 for the strongest style-agnostic pose baseline. Within this backbone and dataset, removing style conditioning increased MAE by 15.6%, compared with 12.8% after removing the convolutional branch and 9.3% after removing the Transformer branch. Because the baselines do not receive the style label and style is partly confounded with routine in the dataset, these results are interpreted as controlled evidence that style information is a major factor, not as a general architectural ranking. The complete system operated at 28.8 FPS with 34.7 ms end-to-end latency, supporting the technical feasibility of style-aware pose sensing for instructor-supervised feedback.