Selected work

Representative Papers

SETA: Scaling Environments for Terminal Agents

Jul 12, 2026

This work addresses the challenge of limited large-scale, diverse, and verifiable training environments for terminal-based reinforcement learning (RL). It introduces SETA, a framework featuring two pipelines—SETA-Synth and SETA-Evol—that enable the first scalable and verifiable automatic generation of terminal RL environments. The framework incorporates a unified verification mechanism and a difficulty-adaptive evolution strategy, yielding SETA-Env, an open dataset comprising over 4,500 tasks. By integrating instruction synthesis, environment construction, and automated validation, the authors train agents using the GRPO algorithm on Qwen3-8B and DeepSeek-V4-Flash models. On Terminal-Bench 2.0, these agents achieve 12% pass@1 (state-of-the-art for 8B models) and 43% pass@1 (a 3% improvement), with pass@5 reaching 58% (a 4% gain).

0 citationsRead paper

LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning

May 31, 2026

This work addresses the challenge of balancing efficiency and accuracy in long-context inference (100K+ tokens) by proposing a general, lightweight context compression method. The approach employs a fine-tuned, lightweight cross-attention scoring layer combined with token-level chunking, top-p token budget allocation, positional reordering, and format-agnostic query parsing. A two-stage fine-tuning strategy—first pretraining on Needle-in-a-Haystack (NIAH) tasks followed by task-specific fine-tuning on multi-hop reasoning and code-related benchmarks—enables significant reduction in prefill costs while maintaining or even surpassing full-context performance. The method matches or outperforms full-context baselines on InfiniteBench Code-Debug, substantially narrows performance gaps on LongBench v2 multi-document reasoning tasks, and demonstrates strong transferability across four models spanning three distinct model families, confirming its generality and effectiveness.

0 citationsRead paper

Test-Time Adaptation via Many-Shot Prompting: Benefits, Limits, and Pitfalls

Mar 06, 2026

This study systematically investigates the effectiveness and limitations of test-time adaptation methods that do not require updating model parameters in open-source large language models. Focusing on many-shot in-context learning (ICL), it integrates dynamic and reinforcement-based ICL prompting strategies to evaluate how the number, ordering, and selection mechanisms of examples influence performance across diverse tasks and model architectures. The findings reveal that many-shot prompting substantially improves performance on structured tasks with high information gain but is highly sensitive to example selection, whereas its benefits are limited in open-ended generation tasks. This work delineates the applicability boundaries and potential risks of prompt-based test-time adaptation, offering both theoretical grounding and practical guidance for real-world deployment.

0 citationsRead paper

Cross-Family Speculative Prefill: Training-Free Long-Context Compression with Small Draft Models

Mar 03, 2026

This work addresses the high prefill overhead incurred by long-context prompts in large language model (LLM) agents, a challenge exacerbated by existing compression methods that rely on same-family small draft models. To overcome this limitation, the authors propose a training-free, cross-model-family prompt compression approach that leverages heterogeneous small draft models—such as Qwen, LLaMA, and DeepSeek—to evaluate token importance via attention mechanisms and guide speculative prefilling. This method is the first to demonstrate the effectiveness of cross-family draft models for prompt compression, thereby eliminating the dependency on same-family architectures. It consistently retains 90%–100% of the original model performance across diverse tasks, with occasional improvements due to denoising effects, while significantly reducing time-to-first-token (TTFT) latency.

0 citationsRead paper

The Limits of Long-Context Reasoning in Automated Bug Fixing

Feb 17, 2026

This study addresses the unclear capability of current large language models (LLMs) in code debugging and patch generation within realistic long-context settings (64k–128k tokens). By constructing controlled experiments using the SWE-bench Verified dataset augmented with a human-curated long-context data pipeline and the mini-SWE-agent framework, we systematically evaluate the automatic repair performance of multiple open- and closed-source models. Our work is the first to disentangle the benefits arising from agent-based task decomposition from the models’ intrinsic long-context reasoning abilities, revealing that existing benchmarks fail to effectively assess the latter. Experimental results show a significant drop in repair success rates under ultra-long contexts—for instance, Qwen3-Coder-30B-A3B achieves only 7% and GPT-5-nano 0%—with most agent successes concentrated within the first 20k tokens, underscoring the practical limitations of current models’ usable context capacity.

0 citationsRead paper

Recent publications

Latest Papers

SETA: Scaling Environments for Terminal Agents

Jul 12, 2026

This work addresses the challenge of limited large-scale, diverse, and verifiable training environments for terminal-based reinforcement learning (RL). It introduces SETA, a framework featuring two pipelines—SETA-Synth and SETA-Evol—that enable the first scalable and verifiable automatic generation of terminal RL environments. The framework incorporates a unified verification mechanism and a difficulty-adaptive evolution strategy, yielding SETA-Env, an open dataset comprising over 4,500 tasks. By integrating instruction synthesis, environment construction, and automated validation, the authors train agents using the GRPO algorithm on Qwen3-8B and DeepSeek-V4-Flash models. On Terminal-Bench 2.0, these agents achieve 12% pass@1 (state-of-the-art for 8B models) and 43% pass@1 (a 3% improvement), with pass@5 reaching 58% (a 4% gain).

0 citationsRead paper

LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning

May 31, 2026

This work addresses the challenge of balancing efficiency and accuracy in long-context inference (100K+ tokens) by proposing a general, lightweight context compression method. The approach employs a fine-tuned, lightweight cross-attention scoring layer combined with token-level chunking, top-p token budget allocation, positional reordering, and format-agnostic query parsing. A two-stage fine-tuning strategy—first pretraining on Needle-in-a-Haystack (NIAH) tasks followed by task-specific fine-tuning on multi-hop reasoning and code-related benchmarks—enables significant reduction in prefill costs while maintaining or even surpassing full-context performance. The method matches or outperforms full-context baselines on InfiniteBench Code-Debug, substantially narrows performance gaps on LongBench v2 multi-document reasoning tasks, and demonstrates strong transferability across four models spanning three distinct model families, confirming its generality and effectiveness.

0 citationsRead paper

Test-Time Adaptation via Many-Shot Prompting: Benefits, Limits, and Pitfalls

Mar 06, 2026

This study systematically investigates the effectiveness and limitations of test-time adaptation methods that do not require updating model parameters in open-source large language models. Focusing on many-shot in-context learning (ICL), it integrates dynamic and reinforcement-based ICL prompting strategies to evaluate how the number, ordering, and selection mechanisms of examples influence performance across diverse tasks and model architectures. The findings reveal that many-shot prompting substantially improves performance on structured tasks with high information gain but is highly sensitive to example selection, whereas its benefits are limited in open-ended generation tasks. This work delineates the applicability boundaries and potential risks of prompt-based test-time adaptation, offering both theoretical grounding and practical guidance for real-world deployment.

0 citationsRead paper

Cross-Family Speculative Prefill: Training-Free Long-Context Compression with Small Draft Models

Mar 03, 2026

This work addresses the high prefill overhead incurred by long-context prompts in large language model (LLM) agents, a challenge exacerbated by existing compression methods that rely on same-family small draft models. To overcome this limitation, the authors propose a training-free, cross-model-family prompt compression approach that leverages heterogeneous small draft models—such as Qwen, LLaMA, and DeepSeek—to evaluate token importance via attention mechanisms and guide speculative prefilling. This method is the first to demonstrate the effectiveness of cross-family draft models for prompt compression, thereby eliminating the dependency on same-family architectures. It consistently retains 90%–100% of the original model performance across diverse tasks, with occasional improvements due to denoising effects, while significantly reducing time-to-first-token (TTFT) latency.

0 citationsRead paper

The Limits of Long-Context Reasoning in Automated Bug Fixing

Feb 17, 2026

This study addresses the unclear capability of current large language models (LLMs) in code debugging and patch generation within realistic long-context settings (64k–128k tokens). By constructing controlled experiments using the SWE-bench Verified dataset augmented with a human-curated long-context data pipeline and the mini-SWE-agent framework, we systematically evaluate the automatic repair performance of multiple open- and closed-source models. Our work is the first to disentangle the benefits arising from agent-based task decomposition from the models’ intrinsic long-context reasoning abilities, revealing that existing benchmarks fail to effectively assess the latter. Experimental results show a significant drop in repair success rates under ultra-long contexts—for instance, Qwen3-Coder-30B-A3B achieves only 7% and GPT-5-nano 0%—with most agent successes concentrated within the first 20k tokens, underscoring the practical limitations of current models’ usable context capacity.

0 citationsRead paper