QvTAD: Differential Relative Attribute Learning for Voice Timbre Attribute Detection

2025-08-21

AI Summary

Voice timbre attribute detection (vTAD) faces two key challenges: high subjectivity in attribute descriptions and severe label imbalance—hindering fine-grained modeling and cross-speaker generalization. To address these, we propose QvTAD: (1) a graph-structured data augmentation strategy leveraging directed acyclic graphs and disjoint-set union to automatically mine high-quality paired speech samples; (2) a relative timbre offset-aware differential attention module that explicitly models attribute-level contrastive relationships; and (3) integration of speaker embeddings from pretrained FACodec, enhanced by differential denoising and contrastive amplification mechanisms. Evaluated on the VCTK-RVA benchmark, QvTAD achieves significant improvements across multiple timbre descriptors over state-of-the-art methods, with particularly pronounced gains in cross-speaker generalization. Our framework establishes a new paradigm for vTAD—interpretable, robust, and scalable.

Technology Category

• Natural Language Processing: Speech
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Voice Timbre Attribute Detection (vTAD) plays a pivotal role in fine-grained timbre modeling for speech generation tasks. However, it remains challenging due to the inherently subjective nature of timbre descriptors and the severe label imbalance in existing datasets. In this work, we present QvTAD, a novel pairwise comparison framework based on differential attention, designed to enhance the modeling of perceptual timbre attributes. To address the label imbalance in the VCTK-RVA dataset, we introduce a graph-based data augmentation strategy that constructs a Directed Acyclic Graph and employs Disjoint-Set Union techniques to automatically mine unobserved utterance pairs with valid attribute comparisons. Our framework leverages speaker embeddings from a pretrained FACodec, and incorporates a Relative Timbre Shift-Aware Differential Attention module. This module explicitly models attribute-specific contrasts between paired utterances via differential denoising and contrast amplification mechanisms. Experimental results on the VCTK-RVA benchmark demonstrate that QvTAD achieves substantial improvements across multiple timbre descriptors, with particularly notable gains in cross-speaker generalization scenarios.

Problem

• Addresses subjective timbre descriptors in voice detection
• Solves severe label imbalance in existing datasets
• Enhances cross-speaker generalization for timbre attributes

Innovation

• Pairwise comparison framework with differential attention
• Graph-based data augmentation using DAG and DSU
• Differential denoising and contrast amplification mechanisms