SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding

2026-04-14

AI Summary

This work addresses the challenge that existing large audio language models struggle to accurately localize short-duration events within long audio recordings, primarily due to the scarcity of training data with precise timestamp annotations and the use of unrealistic evaluation benchmarks. To tackle this, we propose SpotSound, which introduces a novel mechanism to suppress timestamp hallucination through a purpose-designed temporal alignment training objective, significantly enhancing localization accuracy. We further introduce SpotSound-Bench, a challenging benchmark where target events constitute less than 10% of the audio, simulating real-world “needle-in-a-haystack” scenarios. Experiments demonstrate that SpotSound achieves state-of-the-art performance across multiple temporal localization tasks while maintaining robust capabilities on general audio-language understanding. Code, models, and the benchmark are publicly released.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Large Audio-Language Models (ALMs) have recently demonstrated remarkable capabilities in holistic audio understanding, yet they remain unreliable for temporal grounding, i.e., the task of pinpointing exactly when an event occurs within long-form audio. This limitation stems from two factors: training data dominated by clip-level supervision lacking precise timestamps, and benchmarks that fail to simulate real-world scenarios where short events are obscured by dense background sounds. In this paper, we introduce SpotSound, an audio language model designed for grounding audio events. SpotSound incorporates a novel training objective, specifically designed to suppress hallucinated timestamps for events absent from the input. Additionally, we present SpotSound-Bench, a challenging temporal grounding benchmark where target events occupy less than ~10\% of each clip, creating a rigorous `needle-in-a-haystack' evaluation. Experiments demonstrate that SpotSound achieves state-of-the-art results on temporal grounding benchmarks while maintaining robust performance across general downstream audio-language tasks. Code, models and benchmark are released on https://loiesun.github.io/spotsound/

Problem

• temporal grounding
• audio-language models
• long-form audio
• event localization
• timestamp accuracy

Innovation

• temporal grounding
• audio-language model
• hallucination suppression
• fine-grained audio understanding
• needle-in-a-haystack benchmark