Bridge-SR: Schr""odinger Bridge for Efficient SR

2025-01-14

AI Summary

To address high computational complexity, poor reconstruction quality, and slow inference in speech super-resolution (SR) for legacy audio restoration, this paper proposes the first waveform-domain speech SR framework grounded in Schrödinger Bridge (SB) theory, enabling efficient reconstruction from arbitrary low sampling rates up to 48 kHz. We formulate the SB as an optimal transport path from low-fidelity observations to high-fidelity targets, leveraging low-rate waveforms as structural priors. A lightweight score-based architecture—featuring a 1.7M-parameter CNN backbone—is designed, integrated with adaptive noise scheduling, dynamic data scaling, and auxiliary loss optimization. On VCTK, our method achieves an LSD of 0.911 in just four sampling steps, outperforming an eight-step conditional diffusion model (LSD = 0.927) and significantly improving both reconstruction fidelity and inference speed—enabling near real-time, high-quality speech restoration.

Technology Category

• Natural Language Processing: Speech
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Semi-Supervised Learning

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Speech super-resolution (SR), which generates a waveform at a higher sampling rate from its low-resolution version, is a long-standing critical task in speech restoration. Previous works have explored speech SR in different data spaces, but these methods either require additional compression networks or exhibit limited synthesis quality and inference speed. Motivated by recent advances in probabilistic generative models, we present Bridge-SR, a novel and efficient any-to-48kHz SR system in the speech waveform domain. Using tractable Schr""odinger Bridge models, we leverage the observed low-resolution waveform as a prior, which is intrinsically informative for the high-resolution target. By optimizing a lightweight network to learn the score functions from the prior to the target, we achieve efficient waveform SR through a data-to-data generation process that fully exploits the instructive content contained in the low-resolution observation. Furthermore, we identify the importance of the noise schedule, data scaling, and auxiliary loss functions, which further improve the SR quality of bridge-based systems. The experiments conducted on the benchmark dataset VCTK demonstrate the efficiency of our system: (1) in terms of sample quality, Bridge-SR outperforms several strong baseline methods under different SR settings, using a lightweight network backbone (1.7M); (2) in terms of inference speed, our 4-step synthesis achieves better performance than the 8-step conditional diffusion counterpart (LSD: 0.911 vs 0.927). Demo at https://bridge-sr.github.io.

Problem

• Speech Super-Resolution
• Audio Restoration
• Efficiency

Innovation

• Bridge-SR
• Schrödinger Bridge
• Audio Restoration