Huí Sù: Co-constructing a Dual Feedback Apparatus

2026-04-28

AI Summary

This work investigates how humans and intelligent music systems can achieve shared agency and co-creation through recursive feedback mechanisms, moving beyond the limitations of predefined mappings or instrumentalized AI. To this end, we developed two intelligent instruments, Sù and Agentier, which introduce embedded recursive feedback loops simultaneously in the audio domain—via latent-variable feedback based on the RAVE model—and in the control domain—through recurrent neural network–based control signal feedback—coupled with diverse hardware interfaces for real-time interaction. Experimental results demonstrate that the system enables deep entanglement and dynamic negotiation between human and AI at both sonic and control levels, allowing music to emerge organically from the interaction process and establishing a novel paradigm for human-AI collaborative creativity.

Abstract

This performance presents a duet between two intelligent musical instruments, Sù (to trace back; to go upstream) and Agentier (playing on agentic clavier), and their human performers, connected through feedback loops. Rather than treating AI as a tool that responds predictably to input, both systems operate recursively, where past actions continuously influence future behaviour. The Sù operates in the audio space through latent representation. Its performer uses Make Noise 0-series synthesisers and MIDI controllers to work with a neural feedback synthesis system based on a RAVE model, with a latent feedback loop embedded within the model's internal structure. This allows the instrument to remember and reuse its own internal states, influencing ongoing sound generation through its recent sonic history. The Agentier functions in the control space. Its performer interacts with the system using a Roland S-1 synthesiser and Keith McMillen QuNeo touchpad, where control gestures are routed into a recurrent neural network that feeds back into the synthesis process. Through this feedback loop, the system actively shapes the evolution of control signals over time. Contrasting feedback in the audio and control domains, the performance explores shared agency, resistance, and negotiation between humans and intelligent musical systems. Musical phenomena are co-produced through the entangled states of interaction, rather than through pre-existing system configuration or fixed mappings.

Problem

• shared agency
• feedback loops
• intelligent musical systems
• human-AI interaction
• co-creation

Innovation

• recursive feedback
• latent representation
• RAVE model
• shared agency
• intelligent musical instruments