Understanding Musical Predictions With an Embodied Interface for Musical Machine Learning.

Area of Science:

• Human-Computer Interaction
• Music Technology
• Machine Learning in Music

Background:

• Machine learning models for music often lack connection to performance practice and physical gestures.
• Integrating AI into physical instruments can bridge the gap between digital models and live musical expression.

Purpose of the Study:

• To investigate how performers understand and utilize different predictive machine learning models within an embodied interface.
• To explore the impact of sonic and physical feedback from predictive models on musical performance.

Main Methods:

• Introduction of EMPI (Embodied Musical Prediction Interface), a one-dimensional continuous input/output system.
• Utilizing a mixture density recurrent neural network (RNN) to predict performer actions and timing.
• Conducting a controlled study of musical performances using various predictive models and feedback levels.

Main Results:

• Performers showed a preference for models trained on human-sourced data but explored affordances in synthetic and random data models.
• The physical representation of predictions influenced the duration of musical performances.
• Different predictive models offered distinct musical affordances to performers.

Conclusions:

• Constrained musical interfaces can reveal the potential of embodied predictive interactions.
• Generative machine learning models can be effectively integrated into real-time musical performance.
• Experimental evidence supports new understandings of musician interaction with AI in performance.