Structured Transforms for Low-Overhead Quantization of Language Models
本文提出了一种改进的基于Kashin分解的语言模型量化方法，采用结构化的正交变换（如DCT）降低计算成本，并通过贪婪算法保证稳定聚类，提升了量化效率与稳定性。
0 citationsRead paper
Selected work
Representative Papers
Spectral-Informed Neural Networks Outperform Spectral Methods in High-dimensional PDEs
This work addresses the longstanding challenge of solving high-dimensional partial differential equations (PDEs), which has been hindered by the curse of dimensionality: conventional spectral methods suffer from rapidly escalating computational costs, while physics-informed neural networks (PINNs) often lack sufficient accuracy and efficiency. The paper proposes a modified Spectral-Informed Neural Network (Modified SINN) that directly approximates unknown spectral coefficients in the spectral domain by introducing a harmonic-analysis-driven coefficient decay scaling and a basis function embedding mechanism. This approach eliminates the need for spatial derivative computations and substantially reduces memory consumption. The method outperforms sparse-grid spectral methods in moderate dimensions and significantly surpasses PINNs in both steady-state and time-dependent high-dimensional PDE problems, achieving markedly higher accuracy and computational efficiency.
0 citationsRead paper
ImprovEvolve: Ask AlphaEvolve to Improve the Input Solution and Then Improvise
This work proposes a novel large language model (LLM)-guided evolutionary computing paradigm for tackling complex mathematical construction and optimization problems. By reparameterizing evolutionary programs into a modular architecture featuring initialization, refinement, and controllable perturbation mechanisms, the approach substantially reduces the cognitive load on the LLM while enhancing both search efficiency and solution quality. Integrating program synthesis with an iterative optimization scheduling strategy, the method establishes new state-of-the-art results for hexagonal packing with 11, 12, 15, and 16 hexagons; further manual fine-tuning improves outcomes for configurations of 14, 17, and 23 hexagons. Additionally, it achieves a new lower bound of 0.96258 for the second-order autocorrelation inequality, demonstrating its effectiveness and superiority.
0 citationsRead paper
Discrete Bridges for Mutual Information Estimation
This work addresses the challenge of accurately estimating mutual information (MI) between discrete random variables by proposing a novel approach based on Discrete Bridge Matching. For the first time, discrete bridge models are introduced into MI estimation, reframing the problem as a domain translation task. The resulting DBMI estimator integrates generative modeling with information-theoretic techniques, specifically tailored for discrete data. Experimental results demonstrate that the proposed method significantly outperforms existing approaches in both low-dimensional and image-based MI estimation tasks, confirming its effectiveness and strong generalization capability.
0 citationsRead paper
Back to Basics: Revisiting Exploration in Reinforcement Learning for LLM Reasoning via Generative Probabilities
Standard reinforcement learning in large language model inference often leads to a decline in policy entropy, resulting in mode collapse and insufficient output diversity. This work addresses this issue by introducing a novel diagnostic perspective based on the dynamics of generation probabilities and proposes an Advantage Reweighting Mechanism (ARM). ARM dynamically modulates exploration intensity by jointly modeling prompt perplexity and answer confidence within the reward signal, thereby enhancing reasoning path diversity without compromising accuracy. Experimental results demonstrate that the method improves Pass@1 by 5.7% and Pass@32 by 13.9% on Qwen2.5 and DeepSeek, respectively, effectively mitigating entropy collapse and promoting diverse yet correct reasoning capabilities.
0 citationsRead paper
Recent publications
Latest Papers
Structured Transforms for Low-Overhead Quantization of Language Models
本文提出了一种改进的基于Kashin分解的语言模型量化方法，采用结构化的正交变换（如DCT）降低计算成本，并通过贪婪算法保证稳定聚类，提升了量化效率与稳定性。
0 citationsRead paper
Spectral-Informed Neural Networks Outperform Spectral Methods in High-dimensional PDEs
This work addresses the longstanding challenge of solving high-dimensional partial differential equations (PDEs), which has been hindered by the curse of dimensionality: conventional spectral methods suffer from rapidly escalating computational costs, while physics-informed neural networks (PINNs) often lack sufficient accuracy and efficiency. The paper proposes a modified Spectral-Informed Neural Network (Modified SINN) that directly approximates unknown spectral coefficients in the spectral domain by introducing a harmonic-analysis-driven coefficient decay scaling and a basis function embedding mechanism. This approach eliminates the need for spatial derivative computations and substantially reduces memory consumption. The method outperforms sparse-grid spectral methods in moderate dimensions and significantly surpasses PINNs in both steady-state and time-dependent high-dimensional PDE problems, achieving markedly higher accuracy and computational efficiency.
0 citationsRead paper
ImprovEvolve: Ask AlphaEvolve to Improve the Input Solution and Then Improvise
This work proposes a novel large language model (LLM)-guided evolutionary computing paradigm for tackling complex mathematical construction and optimization problems. By reparameterizing evolutionary programs into a modular architecture featuring initialization, refinement, and controllable perturbation mechanisms, the approach substantially reduces the cognitive load on the LLM while enhancing both search efficiency and solution quality. Integrating program synthesis with an iterative optimization scheduling strategy, the method establishes new state-of-the-art results for hexagonal packing with 11, 12, 15, and 16 hexagons; further manual fine-tuning improves outcomes for configurations of 14, 17, and 23 hexagons. Additionally, it achieves a new lower bound of 0.96258 for the second-order autocorrelation inequality, demonstrating its effectiveness and superiority.
0 citationsRead paper
Discrete Bridges for Mutual Information Estimation
This work addresses the challenge of accurately estimating mutual information (MI) between discrete random variables by proposing a novel approach based on Discrete Bridge Matching. For the first time, discrete bridge models are introduced into MI estimation, reframing the problem as a domain translation task. The resulting DBMI estimator integrates generative modeling with information-theoretic techniques, specifically tailored for discrete data. Experimental results demonstrate that the proposed method significantly outperforms existing approaches in both low-dimensional and image-based MI estimation tasks, confirming its effectiveness and strong generalization capability.
0 citationsRead paper
Back to Basics: Revisiting Exploration in Reinforcement Learning for LLM Reasoning via Generative Probabilities
Standard reinforcement learning in large language model inference often leads to a decline in policy entropy, resulting in mode collapse and insufficient output diversity. This work addresses this issue by introducing a novel diagnostic perspective based on the dynamics of generation probabilities and proposes an Advantage Reweighting Mechanism (ARM). ARM dynamically modulates exploration intensity by jointly modeling prompt perplexity and answer confidence within the reward signal, thereby enhancing reasoning path diversity without compromising accuracy. Experimental results demonstrate that the method improves Pass@1 by 5.7% and Pass@32 by 13.9% on Qwen2.5 and DeepSeek, respectively, effectively mitigating entropy collapse and promoting diverse yet correct reasoning capabilities.
0 citationsRead paper