Selected work

Representative Papers

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers

May 20, 2026

This work addresses two finite-sample biases inherent in preconditioned optimizers during language model training: coupling bias between gradient and preconditioner estimates due to minibatch approximation, and systematic bias introduced by nonlinear inversion operations. For the first time, these biases are jointly identified and corrected through a unified single-batch bias-correction framework. The proposed method employs cross-fitting to decouple gradient and preconditioner estimation and incorporates a Delta-method-based variance correction to mitigate nonlinearity-induced bias. Compatible with mainstream optimizers such as AdamW, Sophia, and Shampoo, the approach reduces validation loss by 0.15, 0.07, and 0.11 nats, respectively, in Qwen2.5-0.5B pretraining, while maintaining or improving downstream instruction-tuning performance—demonstrating markedly enhanced training stability and efficiency.

0 citationsRead paper

GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction

May 11, 2026

Detecting heterogeneous, context-sensitive personally identifiable information (PII) across multiple languages and scenarios faces dual challenges of scarce labeled data and privacy risks. This work proposes a lightweight solution by constructing a 0.3B-parameter model based on the GLiNER2 architecture, trained via character-level span annotation and transfer learning. The approach introduces a constraint-driven generation pipeline to synthesize 4,910 multilingual text samples, marking the first integration of constraint-driven synthetic data with an efficient named entity recognition framework to enable accurate cross-lingual extraction of 42 PII categories. Evaluated on the SPY benchmark, the method achieves significantly higher span-level F1 scores than five existing systems, including the OpenAI Privacy Filter, and the model is publicly released to advance open research in PII detection.

0 citationsRead paper

GLiGuard: Schema-Conditioned Classification for LLM Safeguard

May 08, 2026

This work addresses the high latency and poor scalability of existing large language model (LLM) safety guardrails, which typically rely on large autoregressive decoders. The authors propose a non-autoregressive, multitask classification framework based on a 0.3B-parameter bidirectional encoder. By introducing a novel schema-conditioned input design that embeds task instructions and label semantics directly into the input sequence, the model simultaneously evaluates prompt/response safety, refusal detection, 14 fine-grained harm categories, and 11 jailbreaking strategies in a single forward pass. Evaluated across nine safety benchmarks, this approach matches the F1 performance of much larger 7B–27B decoder-based guardrails while reducing model size by 23–90×, achieving up to 16× higher throughput and up to 17× lower latency.

0 citationsRead paper

Pioneer Agent: Continual Improvement of Small Language Models in Production

Apr 10, 2026

This work addresses the engineering bottlenecks that hinder efficient task adaptation of small language models in production, particularly in data curation, error diagnosis, and iterative control. The authors propose the first fully automated closed-loop adaptation system for small language models: during cold-start, it requires only a natural language task description to automatically collect data, construct evaluation sets, and jointly optimize data selection, hyperparameters, and learning strategies; in production, it diagnoses failure patterns from erroneous samples, synthesizes targeted training data, and re-trains the model under explicit regression constraints. This system achieves end-to-end automation from cold-start to continuous deployment without human intervention, autonomously discovering effective strategies such as chain-of-thought supervision and quality-aware data filtering. It yields performance gains of 1.6–83.8 points across eight cold-start tasks and consistently maintains or improves performance across all seven scenarios in the new AdaptFT-Bench benchmark—where baseline methods degrade by up to 43 points—achieving 99.3% accuracy on intent classification and 0.810 F1 on entity recognition.

0 citationsRead paper

Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents

Oct 22, 2025

Existing evaluations of proactivity in LLM-based agents lack systematic, cross-source, long-horizon benchmarks. Method: We propose PROBE, the first formal framework that conceptualizes proactivity as a three-stage capability—problem discovery, bottleneck identification, and solution execution—and introduces a multi-stage evaluation pipeline supporting cross-context reasoning, long-term memory tracking, and action verification. Contribution/Results: PROBE is the first quantifiable, end-to-end benchmark for measuring agent proactivity. Empirical evaluation reveals that state-of-the-art models (e.g., GPT-5, Claude Opus-4.1) achieve only ~40% end-to-end success on realistic proactivity tasks, exposing fundamental deficiencies—including goal drift, context forgetting, and execution fragmentation. PROBE establishes a reproducible, scalable evaluation paradigm and identifies concrete avenues for improvement, thereby advancing rigorous, principled research on proactive LLM agents.

0 citationsRead paper

Recent publications

Latest Papers

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers

May 20, 2026

This work addresses two finite-sample biases inherent in preconditioned optimizers during language model training: coupling bias between gradient and preconditioner estimates due to minibatch approximation, and systematic bias introduced by nonlinear inversion operations. For the first time, these biases are jointly identified and corrected through a unified single-batch bias-correction framework. The proposed method employs cross-fitting to decouple gradient and preconditioner estimation and incorporates a Delta-method-based variance correction to mitigate nonlinearity-induced bias. Compatible with mainstream optimizers such as AdamW, Sophia, and Shampoo, the approach reduces validation loss by 0.15, 0.07, and 0.11 nats, respectively, in Qwen2.5-0.5B pretraining, while maintaining or improving downstream instruction-tuning performance—demonstrating markedly enhanced training stability and efficiency.

0 citationsRead paper

GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction

May 11, 2026

Detecting heterogeneous, context-sensitive personally identifiable information (PII) across multiple languages and scenarios faces dual challenges of scarce labeled data and privacy risks. This work proposes a lightweight solution by constructing a 0.3B-parameter model based on the GLiNER2 architecture, trained via character-level span annotation and transfer learning. The approach introduces a constraint-driven generation pipeline to synthesize 4,910 multilingual text samples, marking the first integration of constraint-driven synthetic data with an efficient named entity recognition framework to enable accurate cross-lingual extraction of 42 PII categories. Evaluated on the SPY benchmark, the method achieves significantly higher span-level F1 scores than five existing systems, including the OpenAI Privacy Filter, and the model is publicly released to advance open research in PII detection.

0 citationsRead paper

GLiGuard: Schema-Conditioned Classification for LLM Safeguard

May 08, 2026

This work addresses the high latency and poor scalability of existing large language model (LLM) safety guardrails, which typically rely on large autoregressive decoders. The authors propose a non-autoregressive, multitask classification framework based on a 0.3B-parameter bidirectional encoder. By introducing a novel schema-conditioned input design that embeds task instructions and label semantics directly into the input sequence, the model simultaneously evaluates prompt/response safety, refusal detection, 14 fine-grained harm categories, and 11 jailbreaking strategies in a single forward pass. Evaluated across nine safety benchmarks, this approach matches the F1 performance of much larger 7B–27B decoder-based guardrails while reducing model size by 23–90×, achieving up to 16× higher throughput and up to 17× lower latency.

0 citationsRead paper

Pioneer Agent: Continual Improvement of Small Language Models in Production

Apr 10, 2026

This work addresses the engineering bottlenecks that hinder efficient task adaptation of small language models in production, particularly in data curation, error diagnosis, and iterative control. The authors propose the first fully automated closed-loop adaptation system for small language models: during cold-start, it requires only a natural language task description to automatically collect data, construct evaluation sets, and jointly optimize data selection, hyperparameters, and learning strategies; in production, it diagnoses failure patterns from erroneous samples, synthesizes targeted training data, and re-trains the model under explicit regression constraints. This system achieves end-to-end automation from cold-start to continuous deployment without human intervention, autonomously discovering effective strategies such as chain-of-thought supervision and quality-aware data filtering. It yields performance gains of 1.6–83.8 points across eight cold-start tasks and consistently maintains or improves performance across all seven scenarios in the new AdaptFT-Bench benchmark—where baseline methods degrade by up to 43 points—achieving 99.3% accuracy on intent classification and 0.810 F1 on entity recognition.

0 citationsRead paper

Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents

Oct 22, 2025

Existing evaluations of proactivity in LLM-based agents lack systematic, cross-source, long-horizon benchmarks. Method: We propose PROBE, the first formal framework that conceptualizes proactivity as a three-stage capability—problem discovery, bottleneck identification, and solution execution—and introduces a multi-stage evaluation pipeline supporting cross-context reasoning, long-term memory tracking, and action verification. Contribution/Results: PROBE is the first quantifiable, end-to-end benchmark for measuring agent proactivity. Empirical evaluation reveals that state-of-the-art models (e.g., GPT-5, Claude Opus-4.1) achieve only ~40% end-to-end success on realistic proactivity tasks, exposing fundamental deficiencies—including goal drift, context forgetting, and execution fragmentation. PROBE establishes a reproducible, scalable evaluation paradigm and identifies concrete avenues for improvement, thereby advancing rigorous, principled research on proactive LLM agents.

0 citationsRead paper