ACE-Cap: Active Evidence Acquisition via Agentic Co-Evolution for Long-Paragraph Fine-Grained Audio Captioning

2026-08-17

AI Summary

This study addresses the issues of detail omission and passive evidence acquisition in long-form audio captioning by proposing the ACE-Cap framework. The method establishes an active evidence retrieval loop through multi-turn interactions between Composer and Instruct models. Furthermore, it introduces an Agentic Co-Evolution mechanism alongside the LOOP-GRPO algorithm to facilitate role协同 evolution and credit assignment across variable-length interactions. By transforming passive generation into an adaptive evidence acquisition process, this work significantly enhances the completeness and accuracy of long-form audio descriptions. Consequently, ACE-Cap establishes a novel paradigm for fine-grained audio understanding, effectively overcoming previous limitations in capturing detailed auditory information within extended temporal contexts.

Abstract

Long-paragraph fine-grained audio captioning requires models to recover diverse acoustic facts while avoiding omissions and unsupported details. However, prevailing captioners remain passive one-shot generators: once a detail is overlooked, they cannot identify the evidence gap, query the audio for targeted information, or decide when sufficient evidence has been collected. We formulate this task as active evidence acquisition and introduce Agentic Co-Evolution for Captioning (ACE-Cap). The framework uses multi-turn interaction between a Composer and an Instruct model to form a closed evidence-acquisition loop. A Captioner first produces an initial description. Conditioned on this description and the interaction history, a text-only Composer asks targeted questions about unresolved acoustic attributes, while an audio-conditioned Instruct model provides grounded answers. The Composer then decides when to terminate and synthesizes the accumulated evidence into a final caption. ACE-Cap trains these roles through a unified gold-to-prediction reward derived from fixed, gold-grounded multiple-choice questions and a frozen caption-only judge. For credit assignment in variable-length interactions, LOOP-GRPO replaces the trajectory-wide scalar advantage with span-aligned signals: leave-one-out contributions of individual questions to the accumulated evidence, a quality-cost utility for stopping, and an evidence-preservation utility for final synthesis. Role-wise warm-up followed by alternating Composer and Instruct optimization keeps each update a well-defined single-policy problem while allowing the roles to co-evolve. ACE-Cap thus turns captioning from passive one-shot generation into an adaptive process that learns what evidence to acquire, when to stop, and how to preserve it in a long-paragraph caption.

Problem

• Long-paragraph fine-grained audio captioning
• Active evidence acquisition
• Passive one-shot generation
• Evidence gap

Innovation

• Active Evidence Acquisition
• Agentic Co-Evolution
• LOOP-GRPO
• Multi-turn Interaction
• Audio Captioning