Selected work

Representative Papers

Factorized Hypothesis Search for Evidence-to-Taxonomy Retrieval

Aug 06, 2026

This work addresses the "retrieval readiness gap"—a performance bottleneck in large-scale classification systems when inputs consist solely of indirect evidence (e.g., table cells) and suffer from semantic ambiguity. To overcome this, the authors propose a factorized hypothesis search mechanism that decomposes semantic interpretation into composable, named dimensional hypotheses. By leveraging structured query generation, parallel multi-hypothesis retrieval, and dimension-level candidate validation, the approach circumvents reliance on free-form text generation. Evaluated on financial taxonomy labeling and the CodiEsp clinical coding task, the method significantly outperforms existing non-oracle approaches, achieving consistent improvements in Recall@1, Mean Reciprocal Rank (MRR), and final accuracy, thereby demonstrating its effectiveness and robustness.

0 citationsRead paper

Protoreasoning in Tiny Transformers

Aug 05, 2026

This study investigates whether small-scale models can achieve genuine general-purpose reasoning rather than relying on heuristic strategies. Focusing on a miniature Transformer with approximately 1 million parameters, the work introduces “protoreasoning”—a streamlined form of chain-of-thought prompting that guides the model to perform explicit, step-by-step reasoning on Dyck language tasks. Experiments demonstrate, for the first time, effective reasoning in a model far below the parameter threshold typically associated with natural language competence, substantially narrowing the out-of-distribution generalization gap. Ablation studies confirm that the performance gains stem from the content of the reasoning trajectories themselves, not merely from increased token count, offering a controllable and interpretable pathway toward understanding the fundamental mechanisms of reasoning.

0 citationsRead paper

Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers

Jul 16, 2026

This work addresses the high memory overhead of recurrent weight-sharing Transformers, which still require separate key/value (K/V) caches for each recurrence step during decoding. The authors propose a post-training cache compression method that exploits the low-rank trajectory structure of recurrent K/V caches across the recurrence dimension. They introduce a two-dimensional latent encoding mechanism that jointly compresses both recurrence and attention-head dimensions while preventing state collapse. The encoder-decoder is optimized via a combination of teacher-initialized SVD, KL divergence regularization, and attention output distillation. Under identical cache budgets, the method substantially outperforms existing approaches: on Ouro-1.4B, it achieves 21.3× compression, increases batch capacity from 32 to 768, and improves MATH-500 reasoning accuracy from 0.43 to 0.66.

0 citationsRead paper

Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters

Jun 29, 2026

This study investigates the source of performance gains from chain-of-thought (CoT) reasoning in large language models: whether they stem from the semantic content of intermediate reasoning steps or merely from the additional computation incurred by generating more tokens. By constructing in-distribution pairs of naturally generated short and long reasoning paths and designing controlled experiments with semantically equivalent but length-varied prompts, the work provides the first combined evidence from natural observation and causal intervention, demonstrating that CoT efficacy arises from verifiable reasoning content rather than sheer length. The authors propose a directed acyclic graph (DAG)-based method to verify semantic equivalence and validate their findings across 25 models and 8 benchmarks, showing that redundant tokens have negligible impact on accuracy, while meaningful reasoning content—preserved even under maximal numerical obfuscation—can amplify performance gains by up to 3.24×.

0 citationsRead paper

OrdRankBen: A Novel Ranking Benchmark for Ordinal Relevance in NLP

Mar 02, 2025

NLP ranking evaluation has long suffered from coarse-grained relevance labeling: binary labels fail to distinguish degrees of relevance, while continuous scores lack explicit ordinal structure, hindering fine-grained discrimination. To address this, we introduce OrdRankBen—the first NLP ranking benchmark explicitly designed for ordinal relevance. Its core innovation lies in structured ordinal relevance labels (e.g., “strongly relevant > moderately relevant > weakly relevant > irrelevant”) and two real-world datasets capturing diverse ordinal label distributions. We employ a hybrid construction methodology combining human annotation with controlled distribution sampling, enabling unified evaluation of ranking-specific LMs, general-purpose LLMs, and dedicated ranking LLMs. Experiments demonstrate that ordinal modeling significantly enhances model sensitivity to subtle relevance distinctions, yielding more precise and robust ranking performance characterization across diverse model architectures.

0 citationsRead paper

Recent publications

Latest Papers

Factorized Hypothesis Search for Evidence-to-Taxonomy Retrieval

Aug 06, 2026

This work addresses the "retrieval readiness gap"—a performance bottleneck in large-scale classification systems when inputs consist solely of indirect evidence (e.g., table cells) and suffer from semantic ambiguity. To overcome this, the authors propose a factorized hypothesis search mechanism that decomposes semantic interpretation into composable, named dimensional hypotheses. By leveraging structured query generation, parallel multi-hypothesis retrieval, and dimension-level candidate validation, the approach circumvents reliance on free-form text generation. Evaluated on financial taxonomy labeling and the CodiEsp clinical coding task, the method significantly outperforms existing non-oracle approaches, achieving consistent improvements in Recall@1, Mean Reciprocal Rank (MRR), and final accuracy, thereby demonstrating its effectiveness and robustness.

0 citationsRead paper

Protoreasoning in Tiny Transformers

Aug 05, 2026

This study investigates whether small-scale models can achieve genuine general-purpose reasoning rather than relying on heuristic strategies. Focusing on a miniature Transformer with approximately 1 million parameters, the work introduces “protoreasoning”—a streamlined form of chain-of-thought prompting that guides the model to perform explicit, step-by-step reasoning on Dyck language tasks. Experiments demonstrate, for the first time, effective reasoning in a model far below the parameter threshold typically associated with natural language competence, substantially narrowing the out-of-distribution generalization gap. Ablation studies confirm that the performance gains stem from the content of the reasoning trajectories themselves, not merely from increased token count, offering a controllable and interpretable pathway toward understanding the fundamental mechanisms of reasoning.

0 citationsRead paper

Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers

Jul 16, 2026

This work addresses the high memory overhead of recurrent weight-sharing Transformers, which still require separate key/value (K/V) caches for each recurrence step during decoding. The authors propose a post-training cache compression method that exploits the low-rank trajectory structure of recurrent K/V caches across the recurrence dimension. They introduce a two-dimensional latent encoding mechanism that jointly compresses both recurrence and attention-head dimensions while preventing state collapse. The encoder-decoder is optimized via a combination of teacher-initialized SVD, KL divergence regularization, and attention output distillation. Under identical cache budgets, the method substantially outperforms existing approaches: on Ouro-1.4B, it achieves 21.3× compression, increases batch capacity from 32 to 768, and improves MATH-500 reasoning accuracy from 0.43 to 0.66.

0 citationsRead paper

Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters

Jun 29, 2026

This study investigates the source of performance gains from chain-of-thought (CoT) reasoning in large language models: whether they stem from the semantic content of intermediate reasoning steps or merely from the additional computation incurred by generating more tokens. By constructing in-distribution pairs of naturally generated short and long reasoning paths and designing controlled experiments with semantically equivalent but length-varied prompts, the work provides the first combined evidence from natural observation and causal intervention, demonstrating that CoT efficacy arises from verifiable reasoning content rather than sheer length. The authors propose a directed acyclic graph (DAG)-based method to verify semantic equivalence and validate their findings across 25 models and 8 benchmarks, showing that redundant tokens have negligible impact on accuracy, while meaningful reasoning content—preserved even under maximal numerical obfuscation—can amplify performance gains by up to 3.24×.

0 citationsRead paper

OrdRankBen: A Novel Ranking Benchmark for Ordinal Relevance in NLP

Mar 02, 2025

NLP ranking evaluation has long suffered from coarse-grained relevance labeling: binary labels fail to distinguish degrees of relevance, while continuous scores lack explicit ordinal structure, hindering fine-grained discrimination. To address this, we introduce OrdRankBen—the first NLP ranking benchmark explicitly designed for ordinal relevance. Its core innovation lies in structured ordinal relevance labels (e.g., “strongly relevant > moderately relevant > weakly relevant > irrelevant”) and two real-world datasets capturing diverse ordinal label distributions. We employ a hybrid construction methodology combining human annotation with controlled distribution sampling, enabling unified evaluation of ranking-specific LMs, general-purpose LLMs, and dedicated ranking LLMs. Experiments demonstrate that ordinal modeling significantly enhances model sensitivity to subtle relevance distinctions, yielding more precise and robust ranking performance characterization across diverse model architectures.

0 citationsRead paper