Selected work

Representative Papers

DiariZen Explained: A Tutorial for the Open Source State-of-the-Art Speaker Diarization Pipeline

Apr 23, 2026

This work addresses the challenges of reproducing and extending DiariZen—an open-source state-of-the-art speaker diarization system—stemming from its cross-library and cross-framework dependencies. We propose the first self-contained, visualizable, and code-aligned modular decomposition of the DiariZen pipeline, structured into seven stages: audio preprocessing, WavLM-Large feature extraction (incorporating structured pruning and layer weighting), Conformer-based backend modeling, powerset classification, VBx clustering, and PLDA scoring. Accompanied by executable scripts and visualization examples, our implementation significantly lowers the barrier to entry for researchers, achieves open-source state-of-the-art performance across multiple benchmarks, and enables fully reproducible experimentation and pedagogical demonstration through comprehensive open-source tutorials.

0 citationsRead paper

TCG CREST System Description for the DISPLACE-M Challenge

Mar 02, 2026

This work addresses the degradation of speaker diarization performance in rural healthcare settings due to acoustic noise, within the context of Track 1 of the DISPLACE-M Challenge. It systematically evaluates the impact of voice activity detection (VAD) and clustering strategies, proposing an agglomerative hierarchical clustering (AHC) approach enhanced with large-window median filtering and several improved spectral clustering variants—SC-adapt, SC-PNA, and SC-MK. The framework integrates ECAPA-TDNN embeddings, WavLM pretrained features, and an end-to-end Diarizen architecture. Compared to the SpeechBrain baseline, the Diarizen system achieves a 39% relative reduction in diarization error rate (DER). The best-performing system attains DERs of 10.37% and 9.21% on the development and evaluation sets, respectively, securing sixth place in Phase I of the challenge.

0 citationsRead paper

The TCG CREST -- RKMVERI Submission for the NCIIPC Startup India AI Grand Challenge

Dec 11, 2025

This work addresses the challenge of robust speaker diarization (SD) in multilingual and code-switched scenarios, where low-resource conditions severely degrade SD performance. We propose a language-agnostic end-to-end SD–ASR–NMT joint pipeline. To enhance SD robustness, we introduce a novel multi-kernel consensus spectral clustering framework that integrates lightweight voice activity detection (VAD), fine-tuned ECAPA-TDNN speaker embeddings, multilingual ASR (Whisper/XLS-R), and neural machine translation, augmented by language identification and rule-based post-processing. To our knowledge, this is the first work to empirically validate the engineering feasibility of full-chain co-optimization of SD–ASR–NMT on real-world multilingual mixed audio. Evaluated on the NCIIPC challenge training set, our system reduces diarization error rate (DER) by 32% over baseline methods, supports Hindi, Tamil, English, and their code-switched combinations, achieves an end-to-end real-time factor <1.8×, and significantly improves cross-lingual generalization and system robustness.

0 citationsRead paper

Leveraging AM and FM Rhythm Spectrograms for Dementia Classification and Assessment

Jun 01, 2025

This study addresses the challenge of modeling long-term rhythmic abnormalities in dementia-related speech. We propose Rhythmic Formant Analysis (RFA), a novel method that constructs AM/FM rhythmic spectrograms to explicitly characterize slow temporal modulations in speech signals. Complementing this, we design handcrafted rhythmic morphological features and introduce a ViT-BERT multimodal fusion paradigm to jointly model the visual structural patterns of rhythmic spectrograms and linguistic semantic information. Experimental results demonstrate that our handcrafted features improve classification accuracy by 14.2% over the eGeMAPs baseline. Moreover, the RFA-based spectrogram fusion achieves a 13.1% gain in classification performance compared to conventional Mel-spectrograms and attains state-of-the-art performance on dementia severity regression. This work establishes a new feature representation and a principled multimodal modeling framework for non-invasive, speech-based dementia screening.

0 citationsRead paper

Causal Structure Discovery for Error Diagnostics of Children's ASR

May 31, 2025

Children’s automatic speech recognition (ASR) performance lags significantly behind adults’ due to intertwined physiological (e.g., vocal tract morphology), cognitive (e.g., immature articulation), and extrinsic factors (e.g., limited vocabulary, environmental noise)—yet conventional analyses treat these in isolation. Method: This work pioneers the integration of causal structure discovery (PC/NOTEARS) and structural causal modeling (SCM) into ASR diagnostics for children, uncovering latent causal pathways such as “age → articulation → recognition errors,” quantifying direct and indirect effects, and conducting counterfactual intervention analysis via fine-tuning Whisper and Wav2Vec 2.0. Contribution/Results: Articulation proficiency is identified as the primary mediator of age-related ASR disparities. Fine-tuning effectively mitigates vocabulary constraints but yields marginal improvement on physiologically grounded acoustic mismatches. The framework demonstrates strong cross-model generalizability, advancing beyond reductionist, isolated-factor attribution paradigms in child ASR research.

0 citationsRead paper

Recent publications

Latest Papers

DiariZen Explained: A Tutorial for the Open Source State-of-the-Art Speaker Diarization Pipeline

Apr 23, 2026

This work addresses the challenges of reproducing and extending DiariZen—an open-source state-of-the-art speaker diarization system—stemming from its cross-library and cross-framework dependencies. We propose the first self-contained, visualizable, and code-aligned modular decomposition of the DiariZen pipeline, structured into seven stages: audio preprocessing, WavLM-Large feature extraction (incorporating structured pruning and layer weighting), Conformer-based backend modeling, powerset classification, VBx clustering, and PLDA scoring. Accompanied by executable scripts and visualization examples, our implementation significantly lowers the barrier to entry for researchers, achieves open-source state-of-the-art performance across multiple benchmarks, and enables fully reproducible experimentation and pedagogical demonstration through comprehensive open-source tutorials.

0 citationsRead paper

TCG CREST System Description for the DISPLACE-M Challenge

Mar 02, 2026

This work addresses the degradation of speaker diarization performance in rural healthcare settings due to acoustic noise, within the context of Track 1 of the DISPLACE-M Challenge. It systematically evaluates the impact of voice activity detection (VAD) and clustering strategies, proposing an agglomerative hierarchical clustering (AHC) approach enhanced with large-window median filtering and several improved spectral clustering variants—SC-adapt, SC-PNA, and SC-MK. The framework integrates ECAPA-TDNN embeddings, WavLM pretrained features, and an end-to-end Diarizen architecture. Compared to the SpeechBrain baseline, the Diarizen system achieves a 39% relative reduction in diarization error rate (DER). The best-performing system attains DERs of 10.37% and 9.21% on the development and evaluation sets, respectively, securing sixth place in Phase I of the challenge.

0 citationsRead paper

The TCG CREST -- RKMVERI Submission for the NCIIPC Startup India AI Grand Challenge

Dec 11, 2025

This work addresses the challenge of robust speaker diarization (SD) in multilingual and code-switched scenarios, where low-resource conditions severely degrade SD performance. We propose a language-agnostic end-to-end SD–ASR–NMT joint pipeline. To enhance SD robustness, we introduce a novel multi-kernel consensus spectral clustering framework that integrates lightweight voice activity detection (VAD), fine-tuned ECAPA-TDNN speaker embeddings, multilingual ASR (Whisper/XLS-R), and neural machine translation, augmented by language identification and rule-based post-processing. To our knowledge, this is the first work to empirically validate the engineering feasibility of full-chain co-optimization of SD–ASR–NMT on real-world multilingual mixed audio. Evaluated on the NCIIPC challenge training set, our system reduces diarization error rate (DER) by 32% over baseline methods, supports Hindi, Tamil, English, and their code-switched combinations, achieves an end-to-end real-time factor <1.8×, and significantly improves cross-lingual generalization and system robustness.

0 citationsRead paper

Leveraging AM and FM Rhythm Spectrograms for Dementia Classification and Assessment

Jun 01, 2025

This study addresses the challenge of modeling long-term rhythmic abnormalities in dementia-related speech. We propose Rhythmic Formant Analysis (RFA), a novel method that constructs AM/FM rhythmic spectrograms to explicitly characterize slow temporal modulations in speech signals. Complementing this, we design handcrafted rhythmic morphological features and introduce a ViT-BERT multimodal fusion paradigm to jointly model the visual structural patterns of rhythmic spectrograms and linguistic semantic information. Experimental results demonstrate that our handcrafted features improve classification accuracy by 14.2% over the eGeMAPs baseline. Moreover, the RFA-based spectrogram fusion achieves a 13.1% gain in classification performance compared to conventional Mel-spectrograms and attains state-of-the-art performance on dementia severity regression. This work establishes a new feature representation and a principled multimodal modeling framework for non-invasive, speech-based dementia screening.

0 citationsRead paper

Causal Structure Discovery for Error Diagnostics of Children's ASR

May 31, 2025

Children’s automatic speech recognition (ASR) performance lags significantly behind adults’ due to intertwined physiological (e.g., vocal tract morphology), cognitive (e.g., immature articulation), and extrinsic factors (e.g., limited vocabulary, environmental noise)—yet conventional analyses treat these in isolation. Method: This work pioneers the integration of causal structure discovery (PC/NOTEARS) and structural causal modeling (SCM) into ASR diagnostics for children, uncovering latent causal pathways such as “age → articulation → recognition errors,” quantifying direct and indirect effects, and conducting counterfactual intervention analysis via fine-tuning Whisper and Wav2Vec 2.0. Contribution/Results: Articulation proficiency is identified as the primary mediator of age-related ASR disparities. Fine-tuning effectively mitigates vocabulary constraints but yields marginal improvement on physiologically grounded acoustic mismatches. The framework demonstrates strong cross-model generalizability, advancing beyond reductionist, isolated-factor attribution paradigms in child ASR research.

0 citationsRead paper