Selected work

Representative Papers

Quantize the Target, Quantize the Drafter: Efficient Inference with Qwen3.5-4B

Jul 05, 2026

This work addresses the challenge of achieving low-latency inference for the Qwen3.5-4B model on resource-constrained NVIDIA A10G GPUs by proposing an efficient approach that integrates quantization with speculative decoding. The method employs quantization-aware distillation to recover model accuracy and introduces a two-stage training strategy to construct a block-wise diffusion draft model specifically tailored for the quantized target model. Furthermore, the draft model is enhanced with quantization and sliding window attention to improve long-context decoding efficiency while maintaining a high token acceptance rate. Experimental results demonstrate that, under strict quality constraints, the proposed method achieves an average speedup of 6.98× over the baseline and secured third place in a related competition.

0 citationsRead paper

Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models

Jun 04, 2026

This work addresses the performance degradation in Mixture-of-Experts (MoE) models during quantization, which often stems from routing instability that causes expert selection drift. To mitigate this issue, the authors propose VSRAQ, a novel post-training quantization method that explicitly models routing consistency as a quantization objective. By jointly optimizing value alignment and structural alignment, VSRAQ incorporates routing logits matching along with mechanisms to preserve expert ranking and Top-k boundaries. This approach effectively alleviates path deviation without introducing additional inference overhead. Evaluated on prominent MoE large language models, VSRAQ significantly enhances expert selection consistency and outperforms baseline methods that focus solely on weight reconstruction or isolated router optimization.

0 citationsRead paper

Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking

May 31, 2026

Traditional retrieval-augmented generation (RAG) systems struggle to support complex reasoning due to intent-agnostic retrieval and fragmented information. This work proposes InSemRAG, a novel framework that enhances knowledge coverage through intent-aware retrieval (IAR) and semantics-preserving chunking (SPC), while dynamically restoring semantic completeness of evidence via an iterative retrieve-and-verify mechanism. By integrating small language models (SLMs) to accelerate inference, the approach achieves efficient, low-latency generation. Experimental results demonstrate that InSemRAG improves F1 by 2.65 points on HotPotQA and boosts accuracy by 1.5 points on FEVER, while reducing latency by a factor of 4.32 compared to Multi-Hop RAG.

0 citationsRead paper

Topology-Aware Layer Pruning for Large Vision-Language Models

Apr 14, 2026

This work addresses the challenge of efficiently pruning large vision-language models (LVLMs), which suffer from high computational and memory costs. Existing pruning methods often fail to identify critical transitional layers that govern the evolution of multimodal representations, leading to significant performance degradation. To overcome this limitation, the study introduces topological data analysis into LVLM pruning for the first time. By modeling hidden states of each layer as point clouds, it employs simplicial complexes and Zigzag persistent homology to characterize the evolution of their topological structures across layers. This enables a quantitative measure of inter-layer topological consistency, which is then used to adaptively preserve essential transitional layers. The proposed method consistently outperforms existing pruning strategies across multiple multimodal benchmarks and maintains superior performance under varying sparsity levels.

0 citationsRead paper

Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models

Apr 14, 2026

This work addresses the limitations of current large language models, whose reasoning performance is constrained by handcrafted static prompts and sensitivity to decoding configurations and task distributions, leading to insufficient generalization and stability. Existing automatic prompt optimization approaches predominantly rely on single-agent local search, making it difficult to jointly optimize prompts and hyperparameters. To overcome this, the paper proposes Agent-GWO, a novel framework that introduces swarm intelligence—specifically, the Grey Wolf Optimizer (GWO)—to this domain for the first time. By leveraging the collaborative guidance of α, β, and δ leader agents in GWO, the method unifies prompt templates and decoding hyperparameters into an evolvable agent configuration, enabling dynamic, global joint optimization within a single framework. Experiments demonstrate consistent and significant improvements in accuracy and robustness across multiple mathematical and mixed-reasoning benchmarks, outperforming existing prompt optimization techniques across diverse large language model backbones.

0 citationsRead paper

Recent publications

Latest Papers

Quantize the Target, Quantize the Drafter: Efficient Inference with Qwen3.5-4B

Jul 05, 2026

This work addresses the challenge of achieving low-latency inference for the Qwen3.5-4B model on resource-constrained NVIDIA A10G GPUs by proposing an efficient approach that integrates quantization with speculative decoding. The method employs quantization-aware distillation to recover model accuracy and introduces a two-stage training strategy to construct a block-wise diffusion draft model specifically tailored for the quantized target model. Furthermore, the draft model is enhanced with quantization and sliding window attention to improve long-context decoding efficiency while maintaining a high token acceptance rate. Experimental results demonstrate that, under strict quality constraints, the proposed method achieves an average speedup of 6.98× over the baseline and secured third place in a related competition.

0 citationsRead paper

Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models

Jun 04, 2026

This work addresses the performance degradation in Mixture-of-Experts (MoE) models during quantization, which often stems from routing instability that causes expert selection drift. To mitigate this issue, the authors propose VSRAQ, a novel post-training quantization method that explicitly models routing consistency as a quantization objective. By jointly optimizing value alignment and structural alignment, VSRAQ incorporates routing logits matching along with mechanisms to preserve expert ranking and Top-k boundaries. This approach effectively alleviates path deviation without introducing additional inference overhead. Evaluated on prominent MoE large language models, VSRAQ significantly enhances expert selection consistency and outperforms baseline methods that focus solely on weight reconstruction or isolated router optimization.

0 citationsRead paper

Efficient RAG with Intent-Aware Retrieval and Semantics-Preserving Chunking

May 31, 2026

Traditional retrieval-augmented generation (RAG) systems struggle to support complex reasoning due to intent-agnostic retrieval and fragmented information. This work proposes InSemRAG, a novel framework that enhances knowledge coverage through intent-aware retrieval (IAR) and semantics-preserving chunking (SPC), while dynamically restoring semantic completeness of evidence via an iterative retrieve-and-verify mechanism. By integrating small language models (SLMs) to accelerate inference, the approach achieves efficient, low-latency generation. Experimental results demonstrate that InSemRAG improves F1 by 2.65 points on HotPotQA and boosts accuracy by 1.5 points on FEVER, while reducing latency by a factor of 4.32 compared to Multi-Hop RAG.

0 citationsRead paper

Topology-Aware Layer Pruning for Large Vision-Language Models

Apr 14, 2026

This work addresses the challenge of efficiently pruning large vision-language models (LVLMs), which suffer from high computational and memory costs. Existing pruning methods often fail to identify critical transitional layers that govern the evolution of multimodal representations, leading to significant performance degradation. To overcome this limitation, the study introduces topological data analysis into LVLM pruning for the first time. By modeling hidden states of each layer as point clouds, it employs simplicial complexes and Zigzag persistent homology to characterize the evolution of their topological structures across layers. This enables a quantitative measure of inter-layer topological consistency, which is then used to adaptively preserve essential transitional layers. The proposed method consistently outperforms existing pruning strategies across multiple multimodal benchmarks and maintains superior performance under varying sparsity levels.

0 citationsRead paper

Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models

Apr 14, 2026

This work addresses the limitations of current large language models, whose reasoning performance is constrained by handcrafted static prompts and sensitivity to decoding configurations and task distributions, leading to insufficient generalization and stability. Existing automatic prompt optimization approaches predominantly rely on single-agent local search, making it difficult to jointly optimize prompts and hyperparameters. To overcome this, the paper proposes Agent-GWO, a novel framework that introduces swarm intelligence—specifically, the Grey Wolf Optimizer (GWO)—to this domain for the first time. By leveraging the collaborative guidance of α, β, and δ leader agents in GWO, the method unifies prompt templates and decoding hyperparameters into an evolvable agent configuration, enabling dynamic, global joint optimization within a single framework. Experiments demonstrate consistent and significant improvements in accuracy and robustness across multiple mathematical and mixed-reasoning benchmarks, outperforming existing prompt optimization techniques across diverse large language model backbones.

0 citationsRead paper