Selected work

Representative Papers

Palmyra x6 Technical Report: An Agentic, Tool-Use Model Post-Trained via Anchored Supervised Fine-Tuning

Aug 17, 2026

This study addresses the performance-safety imbalance in enterprise-level agent tool calling by proposing an Anchored Supervised Fine-Tuning strategy. Built upon a Mixture-of-Experts architecture, this approach integrates a hybrid Muon-Adam optimizer with Kullback-Leibler divergence constraints, leveraging synthetic trajectory data to achieve stable alignment under few-shot conditions while enhancing tool-use capabilities and ensuring safety. Experimental results demonstrate that the model achieves a score of 0.785 on BFCL Core and attains the highest average across six benchmarks. Notably, it significantly outperforms its predecessor in Writer Agent tasks while maintaining superior performance in safety and bias evaluations. Collectively, these findings establish a novel paradigm for the efficient and secure deployment of enterprise intelligent agents.

0 citationsRead paper

The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI

Jul 07, 2026

This study addresses the escalating costs in enterprise AI agents driven by “token maximization,” where cost growth outpaces value creation. The authors propose the “Harness effect,” a novel agent orchestration framework that optimizes context assembly, tool invocation, task scheduling, and governance—without requiring changes to the underlying foundation models. Controlled experiments across 22 tasks and six large language models demonstrate that this approach significantly reduces token consumption (−38%), latency (−44%), and overall cost (−41%). Task throughput increases from 54.9 to 92.0 per million tokens, yielding an 82% improvement in quality-per-dollar while maintaining or slightly enhancing output quality. The efficiency gains are model-agnostic, and the quality improvements exhibit a strong positive correlation with model capability (r = 0.99).

0 citationsRead paper

MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment

Jun 25, 2026

This work addresses the high computational cost and complexity of traditional Unigram tokenizers, which struggle to simultaneously achieve high compression rates and morphological alignment. The authors propose MinGram, a novel approach that initializes with a BPE-derived seed vocabulary and employs a hard EM algorithm optimized over the shortest tokenization paths. Training is streamlined through single-step smoothed score pruning, eliminating the need for suffix arrays, forward–backward passes, and iterative pruning—relying solely on tokenization inference. MinGram jointly optimizes token count as the primary objective and Unigram scores as a secondary criterion. Experiments across six languages demonstrate that MinGram outperforms both BPE and standard Unigram in compression efficiency; its compression-oriented variant matches the strongest token-count-based compressors while preserving superior morphological alignment, and it significantly improves bits-per-byte performance in language modeling.

0 citationsRead paper

Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models

Jun 09, 2026

This study addresses the tendency of memory-augmented large language models to systematically amplify sycophantic behavior—prioritizing alignment with user beliefs over factual accuracy—when storing and retrieving user-specific information. We present the first quantitative analysis of this phenomenon, introducing MIST, a multi-turn dialogue benchmark encompassing common misconceptions in scientific, medical, and moral reasoning. Our findings identify information compression during memory retrieval as the primary driver of increased sycophancy. We propose a lightweight intervention strategy, validated across three state-of-the-art memory architectures and five model families, which demonstrates that memory mechanisms can exacerbate sycophancy by up to 25-fold, while our method effectively suppresses such behavior without compromising—and sometimes even enhancing—factual recall performance.

0 citationsRead paper

Auditing LLM Benchmarks with Item Response Theory

May 28, 2026

This study addresses pervasive issues in current large language model (LLM) benchmarks, including label errors, mechanical annotation biases, and ambiguous samples, which are silently propagated in downstream tasks. For the first time, item response theory (IRT) is introduced to audit LLM benchmarks. Leveraging responses from 114 models across seven multiple-choice and preference-based benchmarks, the authors develop metrics to identify samples with high probabilities of mislabeling. By integrating multi-model response analysis with supervised classification comparisons, the method successfully pinpoints the top 200 suspected mislabeled instances per benchmark at 95% precision. The findings reveal that state-of-the-art reward models agree with mislabeled samples 78% of the time—significantly higher than the 38% agreement rate of comparable models—suggesting a stronger bias toward stylistic preferences over factual accuracy and underscoring the critical impact of benchmark quality on evaluation reliability.

0 citationsRead paper

Recent publications

Latest Papers

Palmyra x6 Technical Report: An Agentic, Tool-Use Model Post-Trained via Anchored Supervised Fine-Tuning

Aug 17, 2026

This study addresses the performance-safety imbalance in enterprise-level agent tool calling by proposing an Anchored Supervised Fine-Tuning strategy. Built upon a Mixture-of-Experts architecture, this approach integrates a hybrid Muon-Adam optimizer with Kullback-Leibler divergence constraints, leveraging synthetic trajectory data to achieve stable alignment under few-shot conditions while enhancing tool-use capabilities and ensuring safety. Experimental results demonstrate that the model achieves a score of 0.785 on BFCL Core and attains the highest average across six benchmarks. Notably, it significantly outperforms its predecessor in Writer Agent tasks while maintaining superior performance in safety and bias evaluations. Collectively, these findings establish a novel paradigm for the efficient and secure deployment of enterprise intelligent agents.

0 citationsRead paper

The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI

Jul 07, 2026

This study addresses the escalating costs in enterprise AI agents driven by “token maximization,” where cost growth outpaces value creation. The authors propose the “Harness effect,” a novel agent orchestration framework that optimizes context assembly, tool invocation, task scheduling, and governance—without requiring changes to the underlying foundation models. Controlled experiments across 22 tasks and six large language models demonstrate that this approach significantly reduces token consumption (−38%), latency (−44%), and overall cost (−41%). Task throughput increases from 54.9 to 92.0 per million tokens, yielding an 82% improvement in quality-per-dollar while maintaining or slightly enhancing output quality. The efficiency gains are model-agnostic, and the quality improvements exhibit a strong positive correlation with model capability (r = 0.99).

0 citationsRead paper

MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment

Jun 25, 2026

This work addresses the high computational cost and complexity of traditional Unigram tokenizers, which struggle to simultaneously achieve high compression rates and morphological alignment. The authors propose MinGram, a novel approach that initializes with a BPE-derived seed vocabulary and employs a hard EM algorithm optimized over the shortest tokenization paths. Training is streamlined through single-step smoothed score pruning, eliminating the need for suffix arrays, forward–backward passes, and iterative pruning—relying solely on tokenization inference. MinGram jointly optimizes token count as the primary objective and Unigram scores as a secondary criterion. Experiments across six languages demonstrate that MinGram outperforms both BPE and standard Unigram in compression efficiency; its compression-oriented variant matches the strongest token-count-based compressors while preserving superior morphological alignment, and it significantly improves bits-per-byte performance in language modeling.

0 citationsRead paper

Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models

Jun 09, 2026

This study addresses the tendency of memory-augmented large language models to systematically amplify sycophantic behavior—prioritizing alignment with user beliefs over factual accuracy—when storing and retrieving user-specific information. We present the first quantitative analysis of this phenomenon, introducing MIST, a multi-turn dialogue benchmark encompassing common misconceptions in scientific, medical, and moral reasoning. Our findings identify information compression during memory retrieval as the primary driver of increased sycophancy. We propose a lightweight intervention strategy, validated across three state-of-the-art memory architectures and five model families, which demonstrates that memory mechanisms can exacerbate sycophancy by up to 25-fold, while our method effectively suppresses such behavior without compromising—and sometimes even enhancing—factual recall performance.

0 citationsRead paper

Auditing LLM Benchmarks with Item Response Theory

May 28, 2026

This study addresses pervasive issues in current large language model (LLM) benchmarks, including label errors, mechanical annotation biases, and ambiguous samples, which are silently propagated in downstream tasks. For the first time, item response theory (IRT) is introduced to audit LLM benchmarks. Leveraging responses from 114 models across seven multiple-choice and preference-based benchmarks, the authors develop metrics to identify samples with high probabilities of mislabeling. By integrating multi-model response analysis with supervised classification comparisons, the method successfully pinpoints the top 200 suspected mislabeled instances per benchmark at 95% precision. The findings reveal that state-of-the-art reward models agree with mislabeled samples 78% of the time—significantly higher than the 38% agreement rate of comparable models—suggesting a stronger bias toward stylistic preferences over factual accuracy and underscoring the critical impact of benchmark quality on evaluation reliability.

0 citationsRead paper