Exploring Sparse Autoencoders in Text-Based Causal Confounding Adjustment
研究使用稀疏自动编码器在基于文本的因果混淆调整中寻找平衡，通过迭代选择最小特征集来解决高维表示与有限样本重叠之间的矛盾。
0 citationsRead paper
Selected work
Representative Papers
What It Costs to Compose, Rebuild, and Correct Precomputed Memory
研究探讨了预计算记忆在语言模型中的正确性保持条件及失效情况，提出通过定期重建和特定格式更新来维持其时效性和准确性。
0 citationsRead paper
Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training
研究通过24次小规模实验测量单个训练样本对预训练模型的影响，使用GPT-2模型和不同条件下的数据注入方法，发现模型从单个样本中学习的信息会随时间衰减。
0 citationsRead paper
A Pilot Study of Autocompleting Tokenizers
This study addresses the computational bottlenecks and performance degradation caused by verbose sequences in byte-level tokenization. We propose a lightweight autoregressive input compression mechanism based on context prediction, which achieves implicit information representation by removing predictable bytes. This approach significantly shortens effective sequence lengths while preserving translation quality. Experiments demonstrate that English-French translation sequences are reduced by nearly one-third without performance loss, while multilingual tasks achieve compression rates of 0.47–0.67 with maintained or improved translation quality. By effectively reducing the computational overhead of byte-level models, this work substantially enhances efficiency in multilingual machine translation.
0 citationsRead paper
Probe-and-Refine Tuning of Repository Guidance for Coding Agents
This work addresses the limitation of current large language model–based coding agents, which struggle to effectively leverage static guidance files—such as AGENTS.md—due to insufficient understanding of high-level repository structure, sometimes even suffering performance degradation. The authors propose a lightweight iterative refinement approach that requires neither agent loops nor external tool calls: it diagnoses issues via synthetic defect-repair probes and progressively refines the guidance file through a single large model invocation per iteration. This study reveals, for the first time, that the generation strategy of guidance files is a critical factor influencing agent performance. Without modifying the underlying model, the method substantially improves repair coverage. On the SWE-bench Verified benchmark, four independent runs using Qwen3.5-35B-A3B achieve an average repair rate of 33.0%, significantly outperforming both a static knowledge base (28.3%) and a no-guidance baseline (25.5%), primarily due to a 14.5-percentage-point increase in coverage.
0 citationsRead paper
Recent publications
Latest Papers
Exploring Sparse Autoencoders in Text-Based Causal Confounding Adjustment
研究使用稀疏自动编码器在基于文本的因果混淆调整中寻找平衡，通过迭代选择最小特征集来解决高维表示与有限样本重叠之间的矛盾。
0 citationsRead paper
What It Costs to Compose, Rebuild, and Correct Precomputed Memory
研究探讨了预计算记忆在语言模型中的正确性保持条件及失效情况，提出通过定期重建和特定格式更新来维持其时效性和准确性。
0 citationsRead paper
Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training
研究通过24次小规模实验测量单个训练样本对预训练模型的影响，使用GPT-2模型和不同条件下的数据注入方法，发现模型从单个样本中学习的信息会随时间衰减。
0 citationsRead paper
A Pilot Study of Autocompleting Tokenizers
This study addresses the computational bottlenecks and performance degradation caused by verbose sequences in byte-level tokenization. We propose a lightweight autoregressive input compression mechanism based on context prediction, which achieves implicit information representation by removing predictable bytes. This approach significantly shortens effective sequence lengths while preserving translation quality. Experiments demonstrate that English-French translation sequences are reduced by nearly one-third without performance loss, while multilingual tasks achieve compression rates of 0.47–0.67 with maintained or improved translation quality. By effectively reducing the computational overhead of byte-level models, this work substantially enhances efficiency in multilingual machine translation.
0 citationsRead paper
Probe-and-Refine Tuning of Repository Guidance for Coding Agents
This work addresses the limitation of current large language model–based coding agents, which struggle to effectively leverage static guidance files—such as AGENTS.md—due to insufficient understanding of high-level repository structure, sometimes even suffering performance degradation. The authors propose a lightweight iterative refinement approach that requires neither agent loops nor external tool calls: it diagnoses issues via synthetic defect-repair probes and progressively refines the guidance file through a single large model invocation per iteration. This study reveals, for the first time, that the generation strategy of guidance files is a critical factor influencing agent performance. Without modifying the underlying model, the method substantially improves repair coverage. On the SWE-bench Verified benchmark, four independent runs using Qwen3.5-35B-A3B achieve an average repair rate of 33.0%, significantly outperforming both a static knowledge base (28.3%) and a no-guidance baseline (25.5%), primarily due to a 14.5-percentage-point increase in coverage.
0 citationsRead paper