Training-Free Model Selection and Domain-Aware Score Calibration for First-Shot Anomalous Sound Detection

2026-07-05

AI Summary

This work addresses the challenges in few-shot anomalous sound detection, where negative correlation between source and target domain AUCs and poor predictability of evaluation performance from development sets hinder effective model assessment. To overcome these issues, the authors propose a training-free post-processing calibration method that applies domain-aware quantile calibration on frozen audio embeddings. The approach incorporates an unsupervised cross-validation criterion for model selection and score calibration, introducing a novel domain-balanced frontier calibration mechanism alongside a label-free domain-balanced selection strategy that relies solely on normal samples, thereby eliminating dependence on labeled development data. Evaluated in the DCASE 2025 challenge, the method improves the evaluation score from 55.83 to 61.05 and achieves a Spearman correlation coefficient of +0.91, ranking 4th retrospectively among 35 participating teams.

Abstract

First-shot anomalous sound detection in DCASE Challenge Task 2 must flag anomalies of unseen machine types with a single threshold, without knowing whether a test clip comes from the data-rich source domain (990 normal training clips) or the data-scarce target domain (10). Two organizer-reported problems remain open: source- and target-domain AUC are negatively correlated across systems, and development-set performance does not predict evaluation-set performance. We address both with a training-free post-hoc layer over frozen audio embeddings: (i) per-domain quantile calibration shrunk toward a pooled map by a prior strength m, tracing a source/target balance frontier, and (ii) a label-free cross-validated domain-balance criterion that ranks candidate configurations from training normals only, paired with a coarse development-labeled viability veto. On DCASE 2025, the criterion rank-predicts the official evaluation score across a 45-configuration grid (Spearman rho = +0.91; family-block bootstrap 95% CI [+0.83, +0.95]) while development score is uninformative (+0.06). Criterion-based selection raises the evaluation score from 55.83 to 59.34 (jackknife CI [2.2, 4.8]) and, on an extended grid, to 61.05 -- retrospectively fourth of 35 teams. Replicating on DCASE 2023 and 2024 bounds the claim: development score is uninformative in all three years and degenerate configurations recur (vetoed every time), but under family-clustered uncertainty the criterion's predictive evidence survives only in 2025; in both replication years a fixed full-equalization default matches or beats criterion-based selection. A DCASE 2026 forward test is frozen before the 2026 evaluation ground truth is released; all headline numbers are reproduced by the official evaluator.

Problem

• anomalous sound detection
• domain calibration
• first-shot learning
• threshold selection
• model selection

Innovation

• training-free
• domain-aware calibration
• first-shot anomaly detection
• quantile calibration
• cross-validated selection