Zero-Shot Temporal Localisation of Audio Deepfakes in Multi-Speaker Conversations

2026-09-09

AI Summary

本文提出一种无监督五阶段流水线方法，解决多说话人对话中音频深度伪造的零样本时间定位问题，使用冻结二元检测器和双阈值解码器。

Abstract

Voice-cloning fraud increasingly relies on surgical injection: a genuine conversation in which only one or two sentences are replaced by synthetic speech. Utterance-level deepfake detectors emit a single real/fake label per clip and cannot report where the synthetic speech lies. We formalise this as Temporal Deepfake Localisation in Multi-Speaker Conversations (TDLMC), show that equal error rate and min-DCF are ill-posed once a file contains both classes, and propose temporal metrics for this regime. Our contribution is a training-free five-stage pipeline that wraps a frozen binary detector and adds segment-level output with no retraining, using a two-threshold hysteresis finitestate-machine decoder to turn noisy window scores into coherent intervals. On 180 constructed multi-speaker conversations from ASVspoof 5, the system attains temporal intersection-over-union 0.90, temporal detection rate 0.95, and MS-DCF 0.26 with a strong backbone, and its false-alarm rate on genuine speech is below 6%, falling under 2% on genuine real multi-speaker dialogue (AMI). Under an identical pipeline, a trained localiser improves temporal IoU by only about 0.04, bounding the cost of forgoing supervision. Evaluated across three frozen detectors under one decoder whose constants are selected on a held-out calibration split, and with a controlled analysis attributing the residual false-alarm rate to a backbone domain gap rather than to the decoder, this provides the first zero-shot baseline and a reusable benchmark for TDLMC.

Problem

• Temporal Deepfake Localisation
• Multi-Speaker Conversations
• Voice-cloning Fraud

Innovation

• zero-shot temporal localisation
• multi-speaker conversations
• hysteresis finite-state machine decoder
• frozen binary detector
• temporal deepfake localisation