Selected work

Representative Papers

Polyglot: Multilingual Style Preserving Speech-Driven Facial Animation

Apr 17, 2026

This work addresses the limitations of existing speech-driven facial animation methods, which predominantly rely on monolingual data and struggle to accommodate linguistic variations and individual speaking styles in multilingual settings. The authors propose a unified diffusion model architecture that implicitly encodes language information through text embeddings and extracts stylistic representations from reference facial sequences, enabling personalized multilingual facial animation without requiring predefined language or speaker labels. Notably, this approach is the first to jointly model the interactive effects of language and speaking style, facilitating cross-lingual and cross-speaker generalization under label-free conditions. Experimental results demonstrate that the method outperforms current state-of-the-art approaches in both monolingual and multilingual scenarios, producing animations that exhibit more natural and realistic articulatory timing, habitual facial gestures, and temporal coherence.

0 citationsRead paper

EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images

Dec 21, 2025

Existing feed-forward 3D Gaussian Splatting (3DGS) methods suffer from uncontrolled Gaussian primitive counts under dense multi-view settings, lacking explicit control over the number of Gaussians during inference. Method: We propose the first feed-forward 3DGS framework enabling explicit specification of the Gaussian count at inference time. Our approach introduces a pixel-aligned initial prediction network and an importance-aware fine-tuning mechanism that jointly performs importance ranking and adaptive parameter rescaling to achieve controllable sparsification; geometric consistency is further enforced via multi-view geometric constraints. Contribution/Results: Under strict constraints on the total number of Gaussians, our method significantly outperforms state-of-the-art approaches in novel-view synthesis quality, while maintaining high reconstruction efficiency and rendering fidelity. This establishes a new paradigm for real-time 3D reconstruction in resource-constrained scenarios.

0 citationsRead paper

Zero-Shot Video Deraining with Video Diffusion Models

Nov 23, 2025

Existing video deraining methods rely on synthetic or static-scene paired data, exhibiting poor generalization; meanwhile, fine-tuning diffusion models often degrades the pretrained generative prior, limiting effectiveness on real-world dynamic rainy scenes. To address this, we propose the first zero-shot video deraining framework—requiring no paired data and performing no model fine-tuning. Leveraging only a pre-trained text-to-video diffusion model, our approach employs latent-space inversion, negative prompting guidance, and a novel attention-switching mechanism to suppress rain streak artifacts while preserving structural consistency of dynamic backgrounds. Extensive experiments demonstrate that our method significantly outperforms prior approaches on real-world rainy videos, achieving superior generalization across diverse dynamic scenarios. This work establishes a new unsupervised paradigm for video deraining in complex, realistic motion-rich environments.

0 citationsRead paper

ProMode: A Speech Prosody Model Conditioned on Acoustic and Textual Inputs

Aug 12, 2025

This work addresses the insufficient fusion of acoustic and textual information in prosody modeling and its poor generalizability to downstream TTS tasks. We propose a standalone end-to-end prosody prediction framework. Methodologically, we design a masked joint encoder that aligns and fuses partially masked acoustic features (e.g., mel-spectrograms) with text sequences to learn fixed-dimensional latent prosodic representations; an encoder-decoder architecture then performs multi-granularity, frame-level prediction of F0 and energy contours. Evaluated on the GigaSpeech dataset, our approach significantly outperforms baselines—including style encoding—in prosody prediction accuracy. When integrated into a TTS system, it improves synthetic speech naturalness and subjective quality, yielding a MOS gain of over 0.3. The framework establishes a new paradigm for disentangled and transferable prosody modeling, enabling robust cross-task prosodic representation learning.

0 citationsRead paper

Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generation

Jun 16, 2025

This work addresses three key challenges in large vision-language models (LVLMs): entanglement of 3D information, coupling between text and geometric control, and embedding noise interference. We propose a 3D representation disentanglement method based on normalized neural tri-planes, integrating the FLAME deformation model and CLIP priors to jointly disentangle text-driven appearance (e.g., age, hairstyle) and 3D geometry (e.g., expression, pose) from unlabeled 2D face images alone. To mitigate LVLM embedding noise and ensure control independence and multi-view consistency, we introduce random Jacobian regularization. To our knowledge, this is the first approach achieving collaborative text–3D disentanglement without paired labels, while preserving identity fidelity and fine-grained detail stability. Our method significantly improves generation quality, controllability, and diversity.

0 citationsRead paper

Recent publications

Latest Papers

Polyglot: Multilingual Style Preserving Speech-Driven Facial Animation

Apr 17, 2026

This work addresses the limitations of existing speech-driven facial animation methods, which predominantly rely on monolingual data and struggle to accommodate linguistic variations and individual speaking styles in multilingual settings. The authors propose a unified diffusion model architecture that implicitly encodes language information through text embeddings and extracts stylistic representations from reference facial sequences, enabling personalized multilingual facial animation without requiring predefined language or speaker labels. Notably, this approach is the first to jointly model the interactive effects of language and speaking style, facilitating cross-lingual and cross-speaker generalization under label-free conditions. Experimental results demonstrate that the method outperforms current state-of-the-art approaches in both monolingual and multilingual scenarios, producing animations that exhibit more natural and realistic articulatory timing, habitual facial gestures, and temporal coherence.

0 citationsRead paper

EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images

Dec 21, 2025

Existing feed-forward 3D Gaussian Splatting (3DGS) methods suffer from uncontrolled Gaussian primitive counts under dense multi-view settings, lacking explicit control over the number of Gaussians during inference. Method: We propose the first feed-forward 3DGS framework enabling explicit specification of the Gaussian count at inference time. Our approach introduces a pixel-aligned initial prediction network and an importance-aware fine-tuning mechanism that jointly performs importance ranking and adaptive parameter rescaling to achieve controllable sparsification; geometric consistency is further enforced via multi-view geometric constraints. Contribution/Results: Under strict constraints on the total number of Gaussians, our method significantly outperforms state-of-the-art approaches in novel-view synthesis quality, while maintaining high reconstruction efficiency and rendering fidelity. This establishes a new paradigm for real-time 3D reconstruction in resource-constrained scenarios.

0 citationsRead paper

Zero-Shot Video Deraining with Video Diffusion Models

Nov 23, 2025

Existing video deraining methods rely on synthetic or static-scene paired data, exhibiting poor generalization; meanwhile, fine-tuning diffusion models often degrades the pretrained generative prior, limiting effectiveness on real-world dynamic rainy scenes. To address this, we propose the first zero-shot video deraining framework—requiring no paired data and performing no model fine-tuning. Leveraging only a pre-trained text-to-video diffusion model, our approach employs latent-space inversion, negative prompting guidance, and a novel attention-switching mechanism to suppress rain streak artifacts while preserving structural consistency of dynamic backgrounds. Extensive experiments demonstrate that our method significantly outperforms prior approaches on real-world rainy videos, achieving superior generalization across diverse dynamic scenarios. This work establishes a new unsupervised paradigm for video deraining in complex, realistic motion-rich environments.

0 citationsRead paper

ProMode: A Speech Prosody Model Conditioned on Acoustic and Textual Inputs

Aug 12, 2025

This work addresses the insufficient fusion of acoustic and textual information in prosody modeling and its poor generalizability to downstream TTS tasks. We propose a standalone end-to-end prosody prediction framework. Methodologically, we design a masked joint encoder that aligns and fuses partially masked acoustic features (e.g., mel-spectrograms) with text sequences to learn fixed-dimensional latent prosodic representations; an encoder-decoder architecture then performs multi-granularity, frame-level prediction of F0 and energy contours. Evaluated on the GigaSpeech dataset, our approach significantly outperforms baselines—including style encoding—in prosody prediction accuracy. When integrated into a TTS system, it improves synthetic speech naturalness and subjective quality, yielding a MOS gain of over 0.3. The framework establishes a new paradigm for disentangled and transferable prosody modeling, enabling robust cross-task prosodic representation learning.

0 citationsRead paper

Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generation

Jun 16, 2025

This work addresses three key challenges in large vision-language models (LVLMs): entanglement of 3D information, coupling between text and geometric control, and embedding noise interference. We propose a 3D representation disentanglement method based on normalized neural tri-planes, integrating the FLAME deformation model and CLIP priors to jointly disentangle text-driven appearance (e.g., age, hairstyle) and 3D geometry (e.g., expression, pose) from unlabeled 2D face images alone. To mitigate LVLM embedding noise and ensure control independence and multi-view consistency, we introduce random Jacobian regularization. To our knowledge, this is the first approach achieving collaborative text–3D disentanglement without paired labels, while preserving identity fidelity and fine-grained detail stability. Our method significantly improves generation quality, controllability, and diversity.

0 citationsRead paper