SCAPES: Semantically Conditioned Autoregressive Prior for Environmental Sounds

2026-09-03

AI Summary

为解决生成环境声音的高成本问题，提出SCAPES模型，采用语义条件自回归先验和连续归一化流方法，在有限资源下高效合成高质量音频。

Abstract

As generative audio models grow in complexity, the computational and ecological costs of synthesizing everyday sounds have become increasingly prohibitive, often requiring industrial-scale resources and massive datasets. In this paper, we present SCAPES: a Semantically Conditioned Autoregressive Prior for Environmental Sounds. SCAPES is a lightweight, resource-efficient generative model designed to synthesize high-fidelity environmental textures through high-level semantic control. By operating on the continuous latent manifold of a neural audio codec, our approach bypasses the rigid structural constraints inherent to discrete tokenization. We propose a segmentation strategy that decomposes audio into overlapping segments, enabling a Continuous Normalizing Flow (CNF) to model the evolution of latent trajectories using Flow Matching. Our experiments demonstrate that a 36-million parameter instance of SCAPES can be trained on limited, uncurated datasets using a single consumer-grade GPU. Notably, convergence is achieved after training for approximately twice the source audio duration, yielding high-fidelity outputs with robust long-term stability and semantic consistency. Furthermore, we showcase the model's capacity for smooth semantic interpolation, providing a flexible and accessible tool for open research and creative sound design. Code, pretrained weights, audio examples, and an interactive demo are publicly available on our project page https://cordutie.github.io/projects/scapes.html

Problem

• generative audio models
• computational costs
• ecological costs
• synthesizing everyday sounds

Innovation

• Semantically Conditioned Autoregressive Prior
• Continuous Normalizing Flow (CNF)
• Flow Matching
• resource-efficient generative model
• latent manifold