🤖 AI Summary
Traditional pitch tracking methods exhibit degraded performance on non-music and non-speech audio—such as bioacoustic and environmental recordings—due to wide bandwidths and rapidly varying fundamental frequencies. To address this, we propose a novel paradigm for pitch contour analysis that bypasses explicit pitch tracking altogether; instead, we model pitch contours as structured visual patterns in time–frequency spectrograms. Leveraging transfer learning, we adapt a natural-image pre-trained object detection architecture (YOLOv5), fine-tuned on synthetically generated pitch contour data, to directly localize and regress pitch trajectories within spectrograms. Evaluated across eight downstream tasks spanning music, speech, bioacoustics, and environmental audio, our method consistently outperforms state-of-the-art pitch trackers—including CREPE and PYIN—demonstrating superior cross-domain generalization and robustness to spectral variability and rapid pitch modulation.
📝 Abstract
This study examines pitch contours as a unifying semantic construct prevalent across various audio domains including music, speech, bioacoustics, and everyday sounds. Analyzing pitch contours offers insights into the universal role of pitch in the perceptual processing of audio signals and contributes to a deeper understanding of auditory mechanisms in both humans and animals. Conventional pitch-tracking methods, while optimized for music and speech, face challenges in handling much broader frequency ranges and more rapid pitch variations found in other audio domains. This study introduces a vision-based approach to pitch contour analysis that eliminates the need for explicit pitch-tracking. The approach uses a convolutional neural network, pre-trained for object detection in natural images and fine-tuned with a dataset of synthetically generated pitch contours, to extract key contour parameters from the time-frequency representation of short audio segments. A diverse set of eight downstream tasks from four audio domains were selected to provide a challenging evaluation scenario for cross-domain pitch contour analysis. The results show that the proposed method consistently surpasses traditional techniques based on pitch-tracking on a wide range of tasks. This suggests that the vision-based approach establishes a foundation for comparative studies of pitch contour characteristics across diverse audio domains.