Elastic Time: Dynamic Frame Rate Bottlenecks for Neural Audio Coding

2026-06-25

AI Summary

Existing neural audio autoencoders, despite supporting variable bitrate coding, are constrained by a fixed latent frame rate, which prevents dynamic adaptation of temporal resolution to the information density of the audio signal, leading to redundancy and inefficiency. This work proposes Elastic Time, a novel mechanism that introduces, for the first time in neural audio coding, a dynamic frame-rate bottleneck. At inference, a lightweight latent predictor skips frames with low information content, while a greedy boundary selection strategy enables accurate reconstruction at the decoder, yielding deployment-time adjustable temporal resolution. Experiments demonstrate that the proposed approach consistently outperforms baseline models across multiple bitrates, achieving a significantly improved efficiency–quality trade-off without compromising audio fidelity, making it well-suited for generative and long-context audio tasks.

Abstract

Neural audio autoencoders have become a core component of compression, feature extraction, and generation. However, while existing systems support variable bitrate, the vast majority of models still operate at a fixed latent frame-rate, allocating equal temporal budget to regions with very different information density, which can result in unnecessarily long sequences. We introduce Elastic Time, a dynamic frame-rate bottleneck that converts fixed-frame-rate autoencoders to dynamic ones. Our method learns a lightweight latent predictor used to decide which frames can be skipped and later reconstructed, enabling efficient greedy boundary selection at inference. Experiments show our method enables deployment-time rate control while improving efficiency-quality tradeoffs relative to baselines. Overall, we provide a flexible mechanism for adjusting temporal resolution in audio autoencoders, potentially facilitating more efficient downstream modeling for generation and long-context tasks.

Problem

• neural audio coding
• fixed frame rate
• temporal resolution
• variable bitrate
• audio autoencoders

Innovation

• dynamic frame rate
• neural audio coding
• latent predictor
• temporal resolution
• autoencoder