Selected work

Representative Papers

ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

Aug 03, 2026

This work addresses the significant storage overhead incurred by retaining the language model head (LM-head) in high-precision formats after weight quantization, as direct quantization severely distorts the logit distribution. To overcome this challenge, the authors propose ARCHead, a novel LM-head compression method that introduces, for the first time, an activation-aware metric combined with low-rank decomposition, grouped INT4 residual quantization, and a low-rank correction mechanism. This approach achieves substantial storage reduction with nearly negligible performance degradation. Evaluated on Qwen3-8B-Base, ARCHead requires only 25.6% of the storage of a BF16 head while achieving a relative perplexity of 1.007. When replacing the original BF16 head, it incurs a minimal cross-entropy increase of merely 0.006–0.007 and reduces throughput by less than 2%.

0 citationsRead paper

Sparse Goodness: How Selective Measurement Transforms Forward-Forward Learning

Mar 28, 2026

This work addresses a key limitation of the traditional Forward-Forward (FF) algorithm, which relies on the sum of squared activations as its goodness function and consequently fails to effectively capture the discriminative structure of neural activity. The study is the first to argue that a goodness function should prioritize the shape of neural responses over their total energy. To this end, it introduces two novel classes of scale-invariant goodness functions: selectivity-based functions (e.g., top-k and entmax-weighted energy) and shape-sensitive functions (e.g., kurtosis and higher-order moments). When integrated with decoupled label-feature forward propagation (FFCL), the proposed approach achieves 98.2% accuracy on MNIST and 89.0% on Fashion-MNIST—improving up to 32.6 percentage points over baseline methods—and consistently outperforms conventional FF across additional benchmarks including USPS and SVHN.

0 citationsRead paper

PAACE: A Plan-Aware Automated Agent Context Engineering Framework

Dec 18, 2025

To address fidelity degradation, attention dilution, and high inference cost in LLM agents caused by context explosion during multi-step planning, this paper proposes PAACE, a plan-aware automated context engineering framework. Methodologically, PAACE introduces explicit planning-structure modeling into context compression for the first time, jointly optimizing task relevance, instruction alignment, and function-call coherence, while introducing PAACE-Syn—a step-level supervised synthetic dataset. Based on this, we develop PAACE-FT, a distillable lightweight compressor. Experiments on AppWorld, OfficeBench, and multi-hop QA benchmarks demonstrate significant improvements in accuracy and F1 score, alongside reductions in peak token count, cumulative dependency, and reasoning steps. Notably, PAACE-FT achieves 97% of teacher-model performance at only one-tenth the inference cost.

0 citationsRead paper

EvoLattice: Persistent Internal-Population Evolution through Multi-Alternative Quality-Diversity Graph Representations for LLM-Guided Program Discovery

Dec 15, 2025

Existing LLM-driven program evolution methods predominantly employ overwrite-style mutation, retaining only a single candidate per iteration—leading to loss of beneficial variants, structural degradation, and poor robustness in fragile search spaces. This paper proposes a directed acyclic graph (DAG)-based multi-alternative population representation: nodes store persistent code fragments, and executable programs correspond to paths through the DAG, enabling structural sharing and combinatorial search. We pioneer implicit encoding of quality-diversity optimization into the graph’s topology; further, we introduce substitution-level statistical evaluation and dependency-aware deterministic self-repair, enhancing structural robustness without compromising LLM-generated flexibility. Experiments demonstrate that our method achieves significantly more stable evolution, greater expressive capacity, and superior performance improvement trajectories on program synthesis and meta-learning tasks—outperforming state-of-the-art LLM-guided evolutionary approaches.

0 citationsRead paper

AI Co-Artist: A LLM-Powered Framework for Interactive GLSL Shader Animation Evolution

Nov 27, 2025

High entry barriers in GLSL shader animation hinder non-programmer artists from engaging in real-time interactive digital art. Method: This paper proposes an LLM-driven, user-guided shader evolution paradigm that integrates GPT-4’s semantic understanding and GLSL program synthesis capabilities within a visual interactive interface, enabling progressive creation via natural language instructions and real-time feedback. Contribution/Results: We introduce user-feedback-driven prompt engineering and multi-round optimization to achieve zero-code, interpretable, and controllable creative generation. Experimental evaluation and user studies demonstrate significant improvements in both创作 efficiency and artistic expressiveness; non-technical users consistently produce professional-grade real-time visual effects. The system exhibits strong generalizability and usability across diverse artistic workflows.

0 citationsRead paper

Recent publications

Latest Papers

ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

Aug 03, 2026

This work addresses the significant storage overhead incurred by retaining the language model head (LM-head) in high-precision formats after weight quantization, as direct quantization severely distorts the logit distribution. To overcome this challenge, the authors propose ARCHead, a novel LM-head compression method that introduces, for the first time, an activation-aware metric combined with low-rank decomposition, grouped INT4 residual quantization, and a low-rank correction mechanism. This approach achieves substantial storage reduction with nearly negligible performance degradation. Evaluated on Qwen3-8B-Base, ARCHead requires only 25.6% of the storage of a BF16 head while achieving a relative perplexity of 1.007. When replacing the original BF16 head, it incurs a minimal cross-entropy increase of merely 0.006–0.007 and reduces throughput by less than 2%.

0 citationsRead paper

Sparse Goodness: How Selective Measurement Transforms Forward-Forward Learning

Mar 28, 2026

This work addresses a key limitation of the traditional Forward-Forward (FF) algorithm, which relies on the sum of squared activations as its goodness function and consequently fails to effectively capture the discriminative structure of neural activity. The study is the first to argue that a goodness function should prioritize the shape of neural responses over their total energy. To this end, it introduces two novel classes of scale-invariant goodness functions: selectivity-based functions (e.g., top-k and entmax-weighted energy) and shape-sensitive functions (e.g., kurtosis and higher-order moments). When integrated with decoupled label-feature forward propagation (FFCL), the proposed approach achieves 98.2% accuracy on MNIST and 89.0% on Fashion-MNIST—improving up to 32.6 percentage points over baseline methods—and consistently outperforms conventional FF across additional benchmarks including USPS and SVHN.

0 citationsRead paper

PAACE: A Plan-Aware Automated Agent Context Engineering Framework

Dec 18, 2025

To address fidelity degradation, attention dilution, and high inference cost in LLM agents caused by context explosion during multi-step planning, this paper proposes PAACE, a plan-aware automated context engineering framework. Methodologically, PAACE introduces explicit planning-structure modeling into context compression for the first time, jointly optimizing task relevance, instruction alignment, and function-call coherence, while introducing PAACE-Syn—a step-level supervised synthetic dataset. Based on this, we develop PAACE-FT, a distillable lightweight compressor. Experiments on AppWorld, OfficeBench, and multi-hop QA benchmarks demonstrate significant improvements in accuracy and F1 score, alongside reductions in peak token count, cumulative dependency, and reasoning steps. Notably, PAACE-FT achieves 97% of teacher-model performance at only one-tenth the inference cost.

0 citationsRead paper

EvoLattice: Persistent Internal-Population Evolution through Multi-Alternative Quality-Diversity Graph Representations for LLM-Guided Program Discovery

Dec 15, 2025

Existing LLM-driven program evolution methods predominantly employ overwrite-style mutation, retaining only a single candidate per iteration—leading to loss of beneficial variants, structural degradation, and poor robustness in fragile search spaces. This paper proposes a directed acyclic graph (DAG)-based multi-alternative population representation: nodes store persistent code fragments, and executable programs correspond to paths through the DAG, enabling structural sharing and combinatorial search. We pioneer implicit encoding of quality-diversity optimization into the graph’s topology; further, we introduce substitution-level statistical evaluation and dependency-aware deterministic self-repair, enhancing structural robustness without compromising LLM-generated flexibility. Experiments demonstrate that our method achieves significantly more stable evolution, greater expressive capacity, and superior performance improvement trajectories on program synthesis and meta-learning tasks—outperforming state-of-the-art LLM-guided evolutionary approaches.

0 citationsRead paper

AI Co-Artist: A LLM-Powered Framework for Interactive GLSL Shader Animation Evolution

Nov 27, 2025

High entry barriers in GLSL shader animation hinder non-programmer artists from engaging in real-time interactive digital art. Method: This paper proposes an LLM-driven, user-guided shader evolution paradigm that integrates GPT-4’s semantic understanding and GLSL program synthesis capabilities within a visual interactive interface, enabling progressive creation via natural language instructions and real-time feedback. Contribution/Results: We introduce user-feedback-driven prompt engineering and multi-round optimization to achieve zero-code, interpretable, and controllable creative generation. Experimental evaluation and user studies demonstrate significant improvements in both创作 efficiency and artistic expressiveness; non-technical users consistently produce professional-grade real-time visual effects. The system exhibits strong generalizability and usability across diverse artistic workflows.

0 citationsRead paper