🤖 AI Summary
This study addresses the high entry barrier, lack of real-time interactivity, and uncontrollable mapping inherent in Chladni pattern–based audiovisual systems in new media art. Leveraging Kirchhoff–Love plate theory, the authors construct an ANSYS-calibrated simulation dataset and propose a lightweight CNN architecture integrated with a CBAM attention mechanism to enable high-precision, low-latency nodal line classification. Building upon this model, they implement an end-to-end real-time mapping system in Python and Max/MSP that, for the first time, achieves controllable, zero-bias, and highly accurate real-time mapping from Chladni patterns to audio frequencies. The system attains a classification accuracy of 99.33%, with an inference latency of 7.03 ms and end-to-end latency under 50 ms, fulfilling real-time interaction requirements and offering a reproducible engineering prototype for new media art applications.
📝 Abstract
In new media art creation, the mapping between vision and hearing is often subjective. As a classic carrier of sound visualization, Chladni patterns have great potential in building audio-visual mapping mechanisms. However, existing tools face pain points: high technical barriers for simulation, offline computing failing real-time interaction, and uncontrollable mapping rules in general sonification tools. To address these, this paper proposes ChladniSonify, a real-time visual-acoustic mapping method for Chladni patterns. Based on Kirchhoff-Love plate theory, we build a paired dataset via numerical programming and calibrate it using ANSYS finite element simulation. Focusing on the slender nodal lines of Chladni patterns, we adopt a lightweight CNN with CBAM to achieve high-precision, low-latency pattern classification. Finally, we build an end-to-end system in Python and Max/MSP, mapping recognized patterns to corresponding sine wave frequencies. Results show the system has excellent usability: the classification module achieves 99.33% accuracy on the test set with 7.03 ms inference latency; the mapped frequency matches the theoretical value with zero deviation; the average end-to-end latency is under 50 ms, meeting real-time interactive needs. This work provides a reproducible engineering prototype for Chladni audio-visual art creation.