Reasoning-Oriented Post-Training and Inference-Time LoRA Rescaling for Audio-Dependent Question Answering

2026-08-24

AI Summary

本文针对音频依赖型问答，通过后训练Low-Rank Adaptation（LoRA）和推理时的LoRA重缩放方法提高模型性能，采用结构化的思维链框架进行分析。

Abstract

Audio-Dependent Question Answering (ADQA) requires Large Audio-Language Models (LALMs) to answer questions whose correct answers depend on the given audio content. Successful ADQA requires accurate audio perception, identification of question-relevant evidence, and cross-modal reasoning. Using the official ADQA dataset of DCASE 2026 Task 5, we investigate reasoning-oriented post-training with Low-Rank Adaptation (LoRA) and inference-time LoRA rescaling for both Qwen2.5-Omni and MOSS-Audio-8B-Thinking. We introduce a structured Chain-of-Thought (CoT) framework that decomposes the reasoning process into question analysis, question type, audio evidence, and reasoning. We then analyze how task-specific LoRA adaptation affects the two backbones and further explore inference-time rescaling of trained LoRA adapters. Experiments on the development set reveal markedly backbone-dependent behavior: post-training improves the Qwen-based systems but substantially degrades MOSS-Audio under our supervised fine-tuning configuration. Moderate LoRA rescaling further improves the best Qwen system's top-1 accuracy from 58.93% to 61.05% and partially restores the performance of the fine-tuned MOSS-Audio models, while the best MOSS-Audio system achieves 67.70% top-1 accuracy. Our submitted systems ranked third overall and second among lightweight systems under 10B parameters in the challenge.

Problem

• Audio-Dependent Question Answering
• Cross-modal Reasoning
• Low-Rank Adaptation (LoRA)
• Chain-of-Thought (CoT)
• Large Audio-Language Models (LALMs)

Innovation

• Low-Rank Adaptation (LoRA)
• Chain-of-Thought (CoT)
• Inference-Time Rescaling
• Audio-Dependent Question Answering (ADQA)