SE-AGCNet: An End-to-End Framework for Joint Speech Enhancement and Loudness Control in Meeting Scenarios

2026-06-24

AI Summary

This work addresses the challenge of severe volume fluctuations in meeting scenarios, where conventional cascaded speech enhancement and automatic gain control (AGC) often lead to noise amplification or excessive speech suppression. To overcome this limitation, we propose SE-AGCNet, the first end-to-end neural framework that jointly optimizes speech enhancement and loudness control. We also introduce a dedicated data generation pipeline, SE-AGC-DataGen, to support model training. The system is trained and evaluated using standardized loudness metrics—namely LUFS, short-term LUFS, and loudness range (LRA)—ensuring precise adherence to target loudness levels while significantly improving both speech quality and automatic speech recognition accuracy. Experimental results demonstrate that our approach consistently outperforms existing baseline methods across all evaluation criteria.

Abstract

Conventional audio pipelines typically treat speech enhancement (SE) and automatic gain control (AGC) as discrete modules, which often limits overall performance. For instance, applying AGC before SE may inadvertently amplify background noise, while prioritizing SE tends to over-suppress low-volume speech. To address these limitations, we propose SE-AGCNet, an end-to-end framework that jointly optimizes SE and AGC. Tailored for meeting scenarios with significant volume variations, SE-AGCNet leverages the synergy between the two tasks: SE preserves quiet speech, thereby facilitating effective volume adjustment by the AGC component. Furthermore, we propose a specialized data simulation pipeline, SE-AGC-DataGen, and incorporate standardized loudness evaluation metrics: integrated loudness (LUFS), short-term loudness (St LUFS), and LRA. Experiments show that SE-AGCNet consistently achieves target loudness while improving speech quality and ASR accuracy over competitive baselines.

Problem

• speech enhancement
• automatic gain control
• meeting scenarios
• loudness control
• end-to-end framework

Innovation

• end-to-end learning
• speech enhancement
• automatic gain control
• loudness normalization
• meeting audio processing