EvMic: Event-Based Non-Contact Sound Recovery From Effective Spatial-Temporal Modeling

When sound waves hit an object, they induce vibrations that produce high-frequency and subtle visual changes, which can be used for recovering the sound. Early studies always encounter trade-offs related to sampling rate, bandwidth, and the simplicity of the optical path. Recent advances in event camera hardware show good potential for its application in visual sound recovery, because of its superior ability in capturing high-frequency signals. Thus we seek to explore the potential of event cameras for non-contact sound recovery. In this work, we present the first learning-based framework for event-based sound recovery, establishing a strong baseline through systematic dataset simulation, hardware design, and model development. Addressing the scarcity of data in this domain, we introduce EvMic, which contains both synthetic and real-world data, enabling data-driven approaches and benchmarking. To effectively capture subtle vibrations in real-world scenarios, we design an imaging system utilizing a laser matrix that enhances surface gradients. Furthermore, we propose a robust end-to-end network that fully exploits the spatial-temporal nature of event streams for high-quality sound recovery. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our proposed framework. The project page is available at https://yyzq1.github.io/EvMic/.