Beyond task performance: Decoding bioacoustic embeddings with speech features

2026-06-12

AI Summary

This study addresses the opacity of acoustic features encoded in pretrained audio embeddings, which hinders their adaptation to rare species or data-scarce scenarios in bioacoustics. For the first time, it systematically evaluates multiple pretrained models across six bioacoustic datasets for their ability to encode the 88-dimensional eGeMAPS acoustic feature set. Combining linear and nonlinear regression probes with normalized mutual information analysis, the work reveals a “no free lunch” phenomenon: different models exhibit distinct feature preferences—e.g., loudness is highly recoverable (R²=0.76), whereas fundamental frequency is markedly harder (R²=0.33). Leveraging feature recoverability and species relevance, the authors propose a data-driven model selection strategy and demonstrate that concatenating embeddings from multiple models yields optimal performance, offering an interpretable and composable guideline for embedding usage in bioacoustic tasks.

Abstract

Pretrained audio embeddings are standard in bioacoustics, yet little is known about which acoustic features these models encode, nor which are useful for a given task. This hinders transparency and limits extension to rare species or data-scarce domains. Here we reveal which speech-like features are encoded in bioacoustic representations. Using the 88~eGeMAPS features across six taxonomic groups, we apply linear and nonlinear regression probes to quantify which acoustic properties each model captures. Results confirm a ``no free lunch'' pattern: no single model captures the full feature space. A concatenated embedding achieves the highest performance, suggesting complementary acoustic space coverage across models. Loudness features are best encoded ($R^2 = 0.76$) while F0 is hardest to recover ($R^2 = 0.33$). By cross-referencing recoverability with per-species feature salience (NMI), we derive data-driven model selection guidance for bioacoustics.

Problem

• bioacoustics
• audio embeddings
• acoustic features
• model transparency
• data-scarce domains

Innovation

• bioacoustic embeddings
• speech features
• probing analysis
• model selection
• eGeMAPS