PeakNetFP: Peak-based Neural Audio Fingerprinting Robust to Extreme Time Stretching

2025-06-26

AI Summary

To address the challenge of balancing robustness and efficiency in audio fingerprinting under extreme temporal stretching (50%–200%), this paper proposes a lightweight neural audio fingerprint system based on spectral peak coordinates. The method innovatively treats sparse spectral peak coordinates directly as 3D point cloud inputs and introduces, for the first time in audio fingerprinting, a PointNet++-style hierarchical point feature extraction network, trained end-to-end via contrastive learning. By abandoning conventional spectrogram representations, the approach reduces input dimensionality by 11× and model parameters to just 1% of NeuralFP’s, while maintaining >90% Top-1 identification accuracy under severe temporal distortions. This yields substantial improvements in both computational efficiency and temporal stretching robustness, establishing a new trade-off frontier for lightweight, distortion-resilient audio fingerprinting.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Learning on the Edge & Model Compression
• Natural Language Processing: Speech

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

This work introduces PeakNetFP, the first neural audio fingerprinting (AFP) system designed specifically around spectral peaks. This novel system is designed to leverage the sparse spectral coordinates typically computed by traditional peak-based AFP methods. PeakNetFP performs hierarchical point feature extraction techniques similar to the computer vision model PointNet++, and is trained using contrastive learning like in the state-of-the-art deep learning AFP, NeuralFP. This combination allows PeakNetFP to outperform conventional AFP systems and achieves comparable performance to NeuralFP when handling challenging time-stretched audio data. In extensive evaluation, PeakNetFP maintains a Top-1 hit rate of over 90% for stretching factors ranging from 50% to 200%. Moreover, PeakNetFP offers significant efficiency advantages: compared to NeuralFP, it has 100 times fewer parameters and uses 11 times smaller input data. These features make PeakNetFP a lightweight and efficient solution for AFP tasks where time stretching is involved. Overall, this system represents a promising direction for future AFP technologies, as it successfully merges the lightweight nature of peak-based AFP with the adaptability and pattern recognition capabilities of neural network-based approaches, paving the way for more scalable and efficient solutions in the field.

Problem

• Develops neural audio fingerprinting robust to extreme time stretching
• Combines peak-based AFP efficiency with neural network adaptability
• Achieves high accuracy with lightweight parameters and small input data

Innovation

• Uses spectral peaks for neural audio fingerprinting
• Applies PointNet++ hierarchical feature extraction
• Employs contrastive learning for robust performance