Related Experiment Video
Updated: Jun 19, 2026

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
From Keystrokes to Auditory Textures: Structural Acoustic Analysis of an Emergent Haptic Soundscape
Abstract:
This paper examines emergent haptic phenomena in distributed mechanically-actuated sound installations through a structural acoustic analysis of Babbling Brook, a large-scale installation comprising forty solenoid-actuated computer keyboards (eighty solenoids in total) and an eight-channel audio keystroke layer. Most haptics research in the arts has focused on direct-contact interfaces and wearable devices designed for individual users. Environmental phenomena produced from haptics have received comparatively little attention, in part because they are difficult to measure directly in walk-through gallery settings. The installation is organized around four discrete density modes that span a perceptual continuum: at low densities listeners perceive discrete mechanical keystrokes, while at high densities the acoustic texture approaches a continuous, broadband noise-like regime reminiscent of rain and water streams. Because direct vibrometric and perceptual measurement were not possible in the gallery setting, we analyze the installation's audio as an acoustic proxy for the underlying haptic transformation, using DSP metrics associated with signal stationarity and auditory-texture perception (envelope smoothness variance, spectral flux, crest factor, waveform kurtosis, spectral spread, and low-frequency modulation). Three vintage computer keyboards with distinct switch mechanisms (rubber dome, Topre capacitive, and Alps clone) are analyzed to examine how mechanical profile shapes the resulting texture, and the same pipeline is applied to three natural water recordings as an external reference. Across modes, four of seven metrics differ with large effect sizes (Kruskal-Wallis $H > 8.6$, $p < 0.035$, $\eta ^{2} > 0.70$). Keyboard-specific profiles that are distinct at sparse densities converge at high densities as waveform statistics approach Gaussian values. Mode IV swarm statistics overlap the natural-water reference on five of seven metrics and are smoother than water on envelope and spectral-variability measures, indicating that the swarm enters the broadband stationary regime of naturalistic water sounds, but lacks water's structured macroscopic fluctuations. Together, these results provide quantitative acoustic evidence supportive of an emergent perceptual transformation from a distributed systems approach to environmental haptics in the arts. Direct vibrotactile measurement and perceptual validation are identified as future work.
Related Concept Videos
Hearing
Perception of Sound Waves
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
Auditory Pathway
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...
Auditory Perception
Sound as Pressure Waves
The pressure fluctuation depends on the difference in displacements between the successive points in the...
Perceiving Loudness, Pitch, and Location
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...
