Interactive Sonification Exploring Emergent Behavior Applying Models for Biological Information and Listening.

Area of Science:

• Data Science
• Human-Computer Interaction
• Acoustic Engineering

Background:

• Sonification, the use of sound to represent data, is challenging for complex, non-linear systems.
• Reproducibility is crucial for sonification, especially when dealing with self-organizing and emergent data behaviors.
• Existing sonification methods lack a formal framework to ensure consistency and facilitate interaction with emergent data.

Purpose of the Study:

• To formalize interactive sonification in an extensible model that supports reproducibility for data exhibiting emergent behavior.
• To develop a framework that integrates reproducible system implementation, interactive multisensory exploration, and reproducible data generation.
• To provide a structured approach for designing and assessing sonification strategies for complex data.

Main Methods:

• Developed an extensible interactive sonification framework with a circular causality signal pathway.
• Incorporated 'tuning functions' between subsystems (Data Source, Sound Generation, Guided Exploration) to manage scale and symmetry.
• Implemented case studies using Chua's circuit and Swarm Chemistry simulations, introducing 'Heuristic Listening' as an assessment model.

Main Results:

• The framework demonstrated performance as a reference model for experimental implementations.
• Successfully identified common sonification structures and functions across different implementations.
• Showcased the impact of affordances on listening scenarios and pattern discovery.

Conclusions:

• The proposed extensible sonification model enhances reproducibility for emergent data.
• Interactive exploration with auditory feedback aids in recognizing complex patterns and dynamics.
• The framework provides a valuable tool for designing and evaluating sonification systems in various applications.