Spatialized Audio in a Custom-Built OpenGL-Based Ear Training Virtual Environment

Interval recognition is an important part of ear training-the key aspect of music education. Once trained, the musician can identify pitches, melodies, chords, and rhythms by listening to music segments. In a conventional setting, the tutor would teach a trainee the intervals using a musical instrument, typically a piano. However, this is expensive, time consuming, and nonengaging for either party. With the emergence of new technologies, including virtual reality (VR) and areas such as edutainment, this and similar trainings can be transformed into more engaging, more accessible, customizable (virtual) environments, with the addition of new cues and bespoke progression settings. In this work, we designed and implemented a VR ear training system for interval recognition. The usability, user experience, and the effect of multimodal integration through the addition of a perceptual cue, spatial audio, was investigated in two experiments with 46 participants. The results show that the system is highly acceptable and provides a very good experience for users. Furthermore, we show that the added spatial auditory cues provided in the VR application give users significantly more information for judging the musical intervals, something that is not possible in a non-VR environment.