🤖 AI Summary
This work addresses critical limitations in current AI music generation systems—namely high latency, unidirectional output, and insufficient role awareness and micro-temporal responsiveness during real-time human–AI improvisational interaction. To overcome these challenges, the authors propose a lightweight, role-aware GRU architecture that unifies modeling of time shifts, harmony, and interaction roles through a multidimensional sparse event stream. They introduce an O(1)-complexity, allocation-free autoregressive decoding mechanism and a three-stage progressive training strategy. Implemented in C++ with JUCE, the system employs lock-free multithreading and integrates the RTNeural inference engine, leveraging compile-time network topology freezing and zero-memory-allocation techniques. Deployed as a DAW plugin, it achieves stable, low-latency human–AI co-performance for the first time, ensuring musical coherence and interactive role dynamics while meeting stringent real-time constraints, thereby establishing a robust and deployable paradigm for AI co-performers.
📝 Abstract
As artificial intelligence advances into the era of Embodied AI, live musical interaction urgently needs to break free from the limitations of offline, unidirectional generation, achieving a "virtual synergy" capable of low-latency, dynamic interplay. To address this, this technical report presents LK_Jam, a real-time, bidirectional human-computer interactive music generation system based on a lightweight Gated Recurrent Unit (GRU) and a high-performance audio host architecture. In the algorithmic representation layer, this system abandons the computationally expensive fixed time-grid. Instead, it constructs a multi-dimensional sparse event stream integrating time-shifts, continuous harmonic embeddings, and role-aware encoding, enabling the model to accurately capture turn-taking logic and micro-timing in a single-step inference. In the engineering implementation layer, this paper builds a strict multithreaded lock-free communication bridge using C++ and the JUCE framework, incorporating the RTNeural inference engine designed specifically for real-time audio. By utilizing compile-time network topology solidification and a zero-allocation (allocation-free) mechanism, the end-to-end overhead of autoregressive decoding is strictly locked at \(O(1)\) complexity, structurally mitigating the risk of audio thread dropouts in DAW plugin environments. Furthermore, this study designs a three-stage progressive training strategy, achieving a leap from basic chord harmonization to expert-level interaction. Preliminary observations and architectural analysis demonstrate that while ensuring musical coherence and interactive role-play, the proposed system successfully challenges extreme real-time engineering constraints, offering a highly robust and deployable technical paradigm for next-generation AI co-performers in live music.