Enhancing Musical Learning Through Mixed Reality: A Case Study Using PocketDrum and Meta Quest 3 for Drum Practice

This work presents a mixed reality application for drum learning that combines the PocketDrum virtual drumming device with the Meta Quest 3 headset, integrating hand tracking to provide an immersive, responsive experience without the need for a physical drum set. The system features a modular architecture for real-time strike detection, visual guidance synchronized with music, spatial calibration, and audio rendering. The system additionally makes use of the headset's color Passthrough during the calibration stage to align the virtual drum kit with the player's position. To evaluate the system's performance, a technical analysis was conducted to measure latency, jitter, and sampling rate across the technologies involved. Additionally, a functional validation experiment assessed how spatial hand tracking from Meta Quest 3 improved PocketDrum's classification accuracy. Results showed that the fused system corrected 19.1% of drum assignment errors made by the inertial-only setup, enhancing consistency in complex rhythmic patterns. These findings demonstrate the effectiveness of sensor fusion for immersive percussion training and support its potential use in accessible, feedback-rich musical learning environments.