🤖 AI Summary
This work addresses the challenge of precise temporal localization in long-form audio recordings—up to 120 minutes—by introducing a time-aware audio large language model. The proposed approach interleaves learnable time tokens periodically within continuous audio token sequences, creating a joint representation that tightly couples temporal information with acoustic content. The model is trained on large-scale synthetic supervision data generated via a cascaded pipeline and employs a mixed-duration training strategy to enhance robustness across varying audio lengths. It achieves high-precision performance on temporally grounded tasks such as time-anchored question answering, segment description, and summarization. Experimental results demonstrate significant improvements over existing methods on both short and long audio benchmarks. To foster further research in audio temporal understanding, the authors have publicly released the model weights and associated datasets.
📝 Abstract
Temporal grounding in long recordings remains challenging for audio-conditioned LLMs. We present a time-aware audio LLM that answers questions with explicit timestamps over up to 120 minutes of input. Our approach interleaves periodic time markers with continuous audio tokens using large-scale synthetic supervision from a cascaded pipeline. Our model achieves strong temporal-grounding accuracy on short and long benchmarks and supports time-anchored fragment descriptions and summaries. Extensive ablations examine how time representation, marker frequency, tokenization, and duration-mixture design affect accuracy and computational cost. We release model weights and datasets to support further research on time-aware audio understanding, available at https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B.