Enhancing Musical Learning Through Mixed Reality: A Case Study Using PocketDrum and Meta Quest 3 for Drum Practice

Area of Science:

• Human-Computer Interaction
• Music Technology
• Virtual Reality

Background:

• Traditional drum learning requires physical instruments and dedicated space.
• Virtual reality (VR) and mixed reality (MR) offer potential for immersive and accessible music education.
• Integrating sensor fusion can enhance the responsiveness and accuracy of virtual musical instruments.

Purpose of the Study:

• To develop and evaluate a mixed reality application for drum learning.
• To combine the PocketDrum virtual drumming device with Meta Quest 3 headset and hand tracking.
• To assess the impact of sensor fusion on drumming accuracy and user experience.

Main Methods:

• Developed a modular MR system with real-time strike detection, visual-audio synchronization, and spatial calibration.
• Utilized Meta Quest 3's color Passthrough for aligning virtual drums.
• Conducted technical analysis for latency, jitter, and sampling rate.
• Performed functional validation to assess hand tracking's impact on classification accuracy.

Main Results:

• The fused system corrected 19.1% of drum assignment errors compared to an inertial-only setup.
• Enhanced consistency in recognizing complex rhythmic patterns.
• Demonstrated the effectiveness of sensor fusion in improving virtual drumming performance.

Conclusions:

• Sensor fusion significantly enhances the accuracy and consistency of virtual drumming.
• The developed MR application provides an immersive and responsive drum learning experience.
• This technology supports accessible, feedback-rich musical education environments.