Selected work

Representative Papers

Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Text-to-Image Generation

Dec 22, 2024arXiv.org

Existing flow-matching models require numerous function evaluations during sampling, compromising the trade-off between efficiency and generation quality—particularly yielding poor consistency in single-step or few-step sampling. This paper proposes a self-correcting flow distillation framework that, for the first time, jointly integrates consistency modeling and adversarial training into the flow-matching paradigm. Leveraging knowledge distillation, our approach enables high-fidelity, highly consistent one-step and few-step text-to-image synthesis. Crucially, it preserves sampling efficiency while substantially improving generation fidelity and stability. Quantitative and qualitative evaluations on CelebA-HQ demonstrate superior performance over state-of-the-art methods. Moreover, zero-shot evaluation on COCO shows significant improvements in text-image alignment and fine-grained detail preservation. The implementation is publicly available.

1 citationsRead paper

Toward a Holistic Approach to Continual Model Merging

Sep 27, 2025

This paper addresses two key challenges in continual model merging—poor scalability and catastrophic forgetting—by proposing the first end-to-end controllable, replay-free framework. Methodologically: (1) prior to merging, tangent-space fine-tuning enhances task-wise weight decoupling; (2) during merging, optimizer states are leveraged to implicitly preserve function-level behavioral information, replacing naive parameter averaging; (3) post-merging, a representation correction mechanism enforces feature-space alignment. The entire pipeline operates with constant memory overhead, requiring neither historical data nor replay buffers. Evaluated on class-incremental and domain-incremental benchmarks, our approach significantly outperforms state-of-the-art methods—reducing forgetting by up to 42%—and scales robustly to hundred-task continual fusion. The framework combines theoretical rigor with practical deployability, offering a principled solution for scalable, memory-efficient continual model integration.

0 citationsRead paper

Few-Shot, No Problem: Descriptive Continual Relation Extraction

Feb 27, 2025

Few-shot continual relation extraction suffers from catastrophic forgetting and difficulty in knowledge consolidation, especially under data scarcity, where modeling relational semantics and mitigating overfitting remain challenging. To address this, we propose the first description-driven retrieval-based continual learning paradigm: leveraging large language models to generate structured relational descriptions, constructing a dual-encoder retrieval framework that jointly encodes class prototypes and semantic descriptions, and designing a reciprocal rank fusion (RRF)-based prediction mechanism for robust inference. Our method achieves significant improvements over state-of-the-art approaches across multiple benchmarks, demonstrating superior stability, strong generalization capability, and effective forgetting mitigation. It establishes a novel paradigm for continual relation learning in low-resource, dynamic environments.

0 citationsRead paper

Few-shot Continual Relation Extraction via Open Information Extraction

Feb 23, 2025

To address three key challenges in few-shot continual relation extraction (FCRE)—catastrophic forgetting, scarcity of labeled instances for novel relations, and difficulty in identifying unseen relations—this paper proposes the first continual learning framework integrating Open Information Extraction (OpenIE) with dynamic Knowledge Graph Construction (KGC). Methodologically, it leverages OpenIE to automatically extract relational triples and incrementally expand the knowledge graph; a few-shot adaptation module enables rapid generalization to new relations, while structural constraints imposed by the graph ensure retention of prior-task knowledge. Innovatively, the framework incorporates open-domain relation discovery into continual learning, supporting zero-shot recognition of unseen relations and enabling dynamic graph evolution. Evaluated on standard FCRE benchmarks, it significantly outperforms state-of-the-art methods, demonstrating superior cross-task knowledge stability and strong generalization capability to previously unobserved relations.

0 citationsRead paper

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts

Feb 05, 2025

This work addresses two key limitations of Low-Rank Adaptation (LoRA): its weak theoretical foundation and the inefficiency of low-rank matrix estimation. We establish, for the first time, a rigorous theoretical interpretation of LoRA from a Mixture of Experts (MoE) perspective, revealing its implicit gating and expert selection mechanisms. Building on this insight, we propose a lightweight, learnable MLP-based reparameterization that replaces conventional fixed-rank LoRA matrices—enabling accelerated low-rank estimation without increasing inference overhead. We theoretically prove that our method reduces the required sample complexity from exponential to polynomial order. Empirically, our approach achieves up to 40.0% performance gain in multi-task few-shot settings; remarkably, it attains comparable performance to fully trained standard LoRA using only 30% of the training data, significantly improving data efficiency and generalization.

0 citationsRead paper

Recent publications

Latest Papers

Toward a Holistic Approach to Continual Model Merging

Sep 27, 2025

This paper addresses two key challenges in continual model merging—poor scalability and catastrophic forgetting—by proposing the first end-to-end controllable, replay-free framework. Methodologically: (1) prior to merging, tangent-space fine-tuning enhances task-wise weight decoupling; (2) during merging, optimizer states are leveraged to implicitly preserve function-level behavioral information, replacing naive parameter averaging; (3) post-merging, a representation correction mechanism enforces feature-space alignment. The entire pipeline operates with constant memory overhead, requiring neither historical data nor replay buffers. Evaluated on class-incremental and domain-incremental benchmarks, our approach significantly outperforms state-of-the-art methods—reducing forgetting by up to 42%—and scales robustly to hundred-task continual fusion. The framework combines theoretical rigor with practical deployability, offering a principled solution for scalable, memory-efficient continual model integration.

0 citationsRead paper

Few-Shot, No Problem: Descriptive Continual Relation Extraction

Feb 27, 2025

Few-shot continual relation extraction suffers from catastrophic forgetting and difficulty in knowledge consolidation, especially under data scarcity, where modeling relational semantics and mitigating overfitting remain challenging. To address this, we propose the first description-driven retrieval-based continual learning paradigm: leveraging large language models to generate structured relational descriptions, constructing a dual-encoder retrieval framework that jointly encodes class prototypes and semantic descriptions, and designing a reciprocal rank fusion (RRF)-based prediction mechanism for robust inference. Our method achieves significant improvements over state-of-the-art approaches across multiple benchmarks, demonstrating superior stability, strong generalization capability, and effective forgetting mitigation. It establishes a novel paradigm for continual relation learning in low-resource, dynamic environments.

0 citationsRead paper

Few-shot Continual Relation Extraction via Open Information Extraction

Feb 23, 2025

To address three key challenges in few-shot continual relation extraction (FCRE)—catastrophic forgetting, scarcity of labeled instances for novel relations, and difficulty in identifying unseen relations—this paper proposes the first continual learning framework integrating Open Information Extraction (OpenIE) with dynamic Knowledge Graph Construction (KGC). Methodologically, it leverages OpenIE to automatically extract relational triples and incrementally expand the knowledge graph; a few-shot adaptation module enables rapid generalization to new relations, while structural constraints imposed by the graph ensure retention of prior-task knowledge. Innovatively, the framework incorporates open-domain relation discovery into continual learning, supporting zero-shot recognition of unseen relations and enabling dynamic graph evolution. Evaluated on standard FCRE benchmarks, it significantly outperforms state-of-the-art methods, demonstrating superior cross-task knowledge stability and strong generalization capability to previously unobserved relations.

0 citationsRead paper

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts

Feb 05, 2025

This work addresses two key limitations of Low-Rank Adaptation (LoRA): its weak theoretical foundation and the inefficiency of low-rank matrix estimation. We establish, for the first time, a rigorous theoretical interpretation of LoRA from a Mixture of Experts (MoE) perspective, revealing its implicit gating and expert selection mechanisms. Building on this insight, we propose a lightweight, learnable MLP-based reparameterization that replaces conventional fixed-rank LoRA matrices—enabling accelerated low-rank estimation without increasing inference overhead. We theoretically prove that our method reduces the required sample complexity from exponential to polynomial order. Empirically, our approach achieves up to 40.0% performance gain in multi-task few-shot settings; remarkably, it attains comparable performance to fully trained standard LoRA using only 30% of the training data, significantly improving data efficiency and generalization.

0 citationsRead paper

On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation

Feb 05, 2025

Zero-initialized attention (ZIA) lacks theoretical justification. Method: We establish, for the first time, a rigorous optimization-theoretic equivalence between ZIA and sparse-gated mixture-of-experts (MoE) models. Building upon this, we propose a unified framework for jointly optimizing linear and nonlinear prompts alongside gating factors; the nonlinear prompt design is provably optimal, enhancing representational capacity and few-shot robustness. Our approach integrates theoretical modeling, co-optimization of gating functions, and architectural adaptation to LLaMA-Adapter. Results: Experiments on open large language model benchmarks demonstrate that nonlinear prompts consistently outperform linear ones; both prompt variants stably surpass standard attention under data scarcity, improving training stability and generalization performance.

0 citationsRead paper