Bagpiper-Edit: Zero-Shot Open-Ended Audio Editing via Rich-Caption

2026-06-19

AI Summary

Existing text-guided audio editing methods rely on paired training data, predefined templates, and separate processing pipelines for speech, music, and general sounds, limiting their capacity for open-domain, free-form editing. This work proposes a novel paradigm based on rich textual description rewriting, which translates user-provided natural language instructions into precise semantic targets while leveraging the original audio as an acoustic context anchor. The approach enables unified, zero-shot editing across diverse audio types without requiring any paired training data. It supports end-to-end manipulation of arbitrary audio content guided by free-form textual instructions, achieving high-fidelity results across speech, music, and general sound editing tasks—performance that rivals that of specialized expert models.

Abstract

Current text-guided audio editing methods rely on paired training data, predefined operation templates, and separate processing pipelines across speech, music, and sound. We present Bagpiper-Edit to enable open-ended audio editing via free-form natural language instructions. We reformulate audio editing as a rich-caption rewriting task by treating a rich caption as the semantic representation of an audio clip. The user request is translated into an edited caption, which then guides Bagpiper-Edit to generate the target edited audio with the original audio as contextual acoustic anchor. This unlocks the potential of free-form editing, and circumvents the need for paired audio-editing training data, enabling powerful zero-shot editing capabilities. Evaluations across speech, audio, and free-form editing show Bagpiper-Edit maintains good consistency to the original audio and achieves similar performance to other expert models in most cases. Demo: https://bagpiper-edit.github.io, Codes: https://github.com/espnet/espnet/pull/6417 & https://github.com/HsunGong/espnet

Problem

• audio editing
• zero-shot
• open-ended
• text-guided
• rich-caption

Innovation

• zero-shot audio editing
• rich-caption rewriting
• open-ended audio editing
• text-guided generation
• acoustic anchor