Selected work

Representative Papers

ShapeGaussian: High-Fidelity 4D Human Reconstruction in Monocular Videos via Vision Priors

Feb 05, 2026

This work addresses the distortions and artifacts commonly observed in existing 4D human reconstruction methods from monocular video, which often stem from reliance on parametric templates such as SMPL or high sensitivity to pose estimation errors. To overcome these limitations, we propose the first template-free, high-fidelity 4D reconstruction framework. Our approach leverages 2D visual priors together with a pretrained data-driven model to generate an initial deformable geometry, which is subsequently refined through a neural deformation field and a multi-reference-frame strategy to capture fine dynamic details. By eliminating template constraints entirely, our method effectively mitigates issues caused by occluded keypoints and inaccurate pose estimates. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art template-based methods in reconstruction accuracy, visual quality, and robustness across diverse everyday motions captured in monocular videos.

0 citationsRead paper

EIR: Enhanced Image Representations for Medical Report Generation

Dec 28, 2025

To address information asymmetry arising from distributional misalignment between visual features and clinical metadata, as well as domain shift when applying natural-image pre-trained models to medical imaging, this paper proposes a cross-modal collaborative modeling framework. First, a medical-domain-specific vision model (e.g., RadImageNet) is employed to extract discriminative X-ray image representations. Second, a cross-modal Transformer architecture is designed to deeply fuse image features with multi-source structured metadata (e.g., age, sex, prior diagnoses). Third, an end-to-end report generation module ensures semantic alignment between visual–metadata inputs and radiology text outputs. Evaluated on MIMIC-CXR and Open-I, the method achieves significant improvements in BLEU-4, CIDEr, and METEOR scores. Blinded evaluations by board-certified radiologists confirm superior clinical relevance and diagnostic accuracy over state-of-the-art approaches. This work is the first to integrate medical pre-trained visual encoders with metadata-aware cross-modal fusion for automated radiology report generation, effectively mitigating both information asymmetry and domain gap challenges.

0 citationsRead paper

Vision-Motion-Reference Alignment for Referring Multi-Object Tracking via Multi-Modal Large Language Models

Nov 21, 2025

Existing Referring Multi-Object Tracking (RMOT) benchmarks provide only static linguistic descriptions—such as appearance, relative position, and initial motion state—failing to capture dynamic motion evolution (e.g., velocity and directional changes), leading to severe temporal misalignment between language and vision modalities and hindering cross-modal tracking performance. To address this, we propose the first MLLM-based RMOT framework, introducing a unified vision–motion–language alignment paradigm. Our approach explicitly models motion as a dedicated modality, designs a hierarchical cross-modal alignment module, and incorporates a motion-guided prediction head to enhance trajectory modeling. Evaluated on multiple RMOT benchmarks, our method achieves significant improvements over state-of-the-art methods, particularly excelling in complex dynamic scenes with superior tracking accuracy and robustness.

0 citationsRead paper

Input-Aware Sparse Attention for Real-Time Co-Speech Video Generation

Oct 02, 2025

Existing diffusion models suffer from slow inference due to multi-step denoising and full-image attention, hindering real-time speech-driven video generation. To address this, we propose a pose-aware video distillation framework. First, we design an input-aware sparse attention mechanism guided by human pose keypoints, focusing computation on facial and hand regions to improve motion coherence. Second, we introduce a pose-conditioned distillation loss to enhance lip-sync accuracy and gesture realism. Third, we employ knowledge distillation to compress a multi-step teacher diffusion model into a computationally efficient few-step student model. Our method achieves real-time inference speed while preserving high visual fidelity. Quantitative and qualitative evaluations demonstrate superior performance over state-of-the-art audio- and input-driven methods in lip-sync precision, hand-motion naturalness, and temporal consistency.

0 citationsRead paper

Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective

Jun 03, 2025

This paper addresses the pervasive “zero-reward assumption” challenge in LLM reinforcement learning—namely, the difficulty of obtaining token-level immediate rewards, leaving only sparse, response-level rewards. We introduce the Trajectory Policy Gradient Theorem, which rigorously proves for the first time that response-level rewards can yield unbiased estimates of the true token-level policy gradient. Methodologically, we establish a mathematical equivalence between response-level rewards and token-level gradients, revealing that mainstream algorithms—including PPO and GRPO—are inherently compatible with this setting. Leveraging this insight, we propose TRePO, a lightweight, memory-efficient algorithm. Key contributions include: (1) providing a unified theoretical foundation for response-level RL; (2) substantially reducing reward engineering overhead in LLM alignment; and (3) achieving competitive performance with simplified implementation and improved training efficiency.

0 citationsRead paper

Recent publications

Latest Papers

ShapeGaussian: High-Fidelity 4D Human Reconstruction in Monocular Videos via Vision Priors

Feb 05, 2026

This work addresses the distortions and artifacts commonly observed in existing 4D human reconstruction methods from monocular video, which often stem from reliance on parametric templates such as SMPL or high sensitivity to pose estimation errors. To overcome these limitations, we propose the first template-free, high-fidelity 4D reconstruction framework. Our approach leverages 2D visual priors together with a pretrained data-driven model to generate an initial deformable geometry, which is subsequently refined through a neural deformation field and a multi-reference-frame strategy to capture fine dynamic details. By eliminating template constraints entirely, our method effectively mitigates issues caused by occluded keypoints and inaccurate pose estimates. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art template-based methods in reconstruction accuracy, visual quality, and robustness across diverse everyday motions captured in monocular videos.

0 citationsRead paper

EIR: Enhanced Image Representations for Medical Report Generation

Dec 28, 2025

To address information asymmetry arising from distributional misalignment between visual features and clinical metadata, as well as domain shift when applying natural-image pre-trained models to medical imaging, this paper proposes a cross-modal collaborative modeling framework. First, a medical-domain-specific vision model (e.g., RadImageNet) is employed to extract discriminative X-ray image representations. Second, a cross-modal Transformer architecture is designed to deeply fuse image features with multi-source structured metadata (e.g., age, sex, prior diagnoses). Third, an end-to-end report generation module ensures semantic alignment between visual–metadata inputs and radiology text outputs. Evaluated on MIMIC-CXR and Open-I, the method achieves significant improvements in BLEU-4, CIDEr, and METEOR scores. Blinded evaluations by board-certified radiologists confirm superior clinical relevance and diagnostic accuracy over state-of-the-art approaches. This work is the first to integrate medical pre-trained visual encoders with metadata-aware cross-modal fusion for automated radiology report generation, effectively mitigating both information asymmetry and domain gap challenges.

0 citationsRead paper

Vision-Motion-Reference Alignment for Referring Multi-Object Tracking via Multi-Modal Large Language Models

Nov 21, 2025

Existing Referring Multi-Object Tracking (RMOT) benchmarks provide only static linguistic descriptions—such as appearance, relative position, and initial motion state—failing to capture dynamic motion evolution (e.g., velocity and directional changes), leading to severe temporal misalignment between language and vision modalities and hindering cross-modal tracking performance. To address this, we propose the first MLLM-based RMOT framework, introducing a unified vision–motion–language alignment paradigm. Our approach explicitly models motion as a dedicated modality, designs a hierarchical cross-modal alignment module, and incorporates a motion-guided prediction head to enhance trajectory modeling. Evaluated on multiple RMOT benchmarks, our method achieves significant improvements over state-of-the-art methods, particularly excelling in complex dynamic scenes with superior tracking accuracy and robustness.

0 citationsRead paper

Input-Aware Sparse Attention for Real-Time Co-Speech Video Generation

Oct 02, 2025

Existing diffusion models suffer from slow inference due to multi-step denoising and full-image attention, hindering real-time speech-driven video generation. To address this, we propose a pose-aware video distillation framework. First, we design an input-aware sparse attention mechanism guided by human pose keypoints, focusing computation on facial and hand regions to improve motion coherence. Second, we introduce a pose-conditioned distillation loss to enhance lip-sync accuracy and gesture realism. Third, we employ knowledge distillation to compress a multi-step teacher diffusion model into a computationally efficient few-step student model. Our method achieves real-time inference speed while preserving high visual fidelity. Quantitative and qualitative evaluations demonstrate superior performance over state-of-the-art audio- and input-driven methods in lip-sync precision, hand-motion naturalness, and temporal consistency.

0 citationsRead paper

Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective

Jun 03, 2025

This paper addresses the pervasive “zero-reward assumption” challenge in LLM reinforcement learning—namely, the difficulty of obtaining token-level immediate rewards, leaving only sparse, response-level rewards. We introduce the Trajectory Policy Gradient Theorem, which rigorously proves for the first time that response-level rewards can yield unbiased estimates of the true token-level policy gradient. Methodologically, we establish a mathematical equivalence between response-level rewards and token-level gradients, revealing that mainstream algorithms—including PPO and GRPO—are inherently compatible with this setting. Leveraging this insight, we propose TRePO, a lightweight, memory-efficient algorithm. Key contributions include: (1) providing a unified theoretical foundation for response-level RL; (2) substantially reducing reward engineering overhead in LLM alignment; and (3) achieving competitive performance with simplified implementation and improved training efficiency.

0 citationsRead paper