What Do Audio-Visual Synchronization Metrics Actually Measure?

📅 2026-08-25
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
本文通过共同审计多种音频-视觉同步度量方法,揭示了它们在测量内容上的差异,并建议采用可靠性卡片报告而非单一分数。
📝 Abstract
Automatic AV-sync metrics are widely used to rank and train audio-visual generators, but they are rarely audited as measurement instruments. We jointly audit AV-Align, ImageBind AV-relevance, JavisScore, and Synchformer/DeSync under a common reliability protocol: controlled-distortion monotonicity, preprocessing sensitivity, rank uncertainty, cross-metric agreement, PEAVS-proxy agreement, and learned fusion. The result is an axis split, not a single winner: Synchformer/DeSync is the strongest temporal-offset tracker ($τ=0.84$), ImageBind/JavisScore better match the PEAVS human-aligned proxy ($τ=0.20$) and content-disruption families, and AV-Align is the weakest standalone metric. The metrics mutually disagree (Krippendorff $α=0.066$), and neither linear nor simple $k$-NN fusion improves PEAVS agreement over the best individual metric. We recommend reporting AV-sync as a Reliability Card (metric-family breakdowns with confidence intervals) rather than a single bare synchronization score.
Problem

Research questions and friction points this paper is trying to address.

AV-sync metrics
reliability
audio-visual generators
metric agreement
Reliability Card
Innovation

Methods, ideas, or system contributions that make the work stand out.

audio-visual synchronization
metric audit
reliability protocol
temporal-offset tracking
human-aligned proxy
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
J
Jai Kumar Sharma
Virginia Tech
P
Peeyush Tapadiya
Accenture