Selected work

Representative Papers

Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Apr 10, 2025Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning

Large language models (LLMs) suffer from low data efficiency, typically requiring trillion-word corpora for effective pretraining. Method: Inspired by child language acquisition, this work proposes a cognitively grounded, highly efficient pretraining paradigm using only a developmentally appropriate corpus of under 100 million tokens. We systematically demonstrate—contrary to prevailing assumptions—that such small-scale data can surpass trillion-parameter models’ performance when combined with short-sequence training, knowledge distillation, and multi-task evaluation (covering syntactic competence, downstream task transfer, and out-of-distribution generalization); notably, curriculum learning proves ineffective in this low-data regime. Contribution/Results: Leveraging the LTG-BERT architecture, our best-performing model achieves state-of-the-art results across diverse benchmarks, significantly outperforming standard large baselines. The project yields over 30 empirically validated guidelines—identifying both viable strategies and dead ends—for efficient pretraining, thereby establishing a novel paradigm for cognitive modeling and environmentally sustainable (“green”) AI.

105 citations18 influentialRead paper

Optimizing the Decoding Probability and Coverage Ratio of Composite DNA

Jul 07, 2024International Symposium on Information Theory

This work addresses two core challenges in the emerging paradigm of composite DNA data storage: (1) determining the expected sequencing depth required to decode individual or multiple composite DNA strands, and (2) designing optimal hybrid symbol sets—under a fixed nucleotide alphabet constraint—to maximize maximum-likelihood decoding success probability. We formally formulate the joint optimization of decoding probability and coverage performance for the first time. A novel information-theoretic framework for hybrid symbol selection is proposed, accompanied by an asymptotic coverage analysis model that yields a closed-form expression for the coverage ratio as a function of read length and hybrid encoding strategy. Our theoretical analysis derives tight probabilistic bounds, substantially enhancing decoding robustness. The results provide computationally tractable and experimentally verifiable design principles for co-optimizing error-correcting codes and sequencing protocols in high-reliability DNA-based archival storage systems.

8 citationsRead paper

CtD: Composition through Decomposition in Emergent Communication

Jan 15, 2026International Conference on Learning Representations

This work addresses the challenge of enabling neural agents to achieve compositional generalization on unseen images within emergent communication. The authors propose a novel "composition through decomposition" paradigm: in a multi-agent coordination game, agents first learn to disentangle images into basic concepts via a learnable codebook, which is then used to compositionally generate complex descriptions. Employing a two-stage sequential training strategy, this approach achieves zero-shot compositional generalization within the emergent communication framework without requiring additional training. Experimental results demonstrate that the agents can effectively describe previously unseen images and exhibit strong compositional generalization capabilities in certain scenarios.

2 citationsRead paper

Will it Merge? On The Causes of Model Mergeability

Jan 10, 2026arXiv.org

Model merging often suffers from unpredictable performance, limiting its practical utility. This work introduces the first quantifiable definition of model mergeability and systematically investigates the key factors influencing merging effectiveness, identifying the base model’s prior knowledge about the fine-tuning data as the decisive factor. Building on this insight, the authors propose a weighted parameter fusion strategy that effectively preserves weak yet relevant knowledge embedded in the base model. Experimental results demonstrate that the proposed method significantly enhances merging performance in multi-task settings, thereby validating the critical role of the base model’s knowledge level in determining the success of model merging.

2 citationsRead paper

Gradient-Variation Online Adaptivity for Accelerated Optimization with H""older Smoothness

Nov 04, 2025

This paper addresses online learning and offline strongly convex optimization under Hölder smoothness, where the smoothness parameters are unknown. To overcome this challenge, we propose the first fully adaptive online algorithm: it employs a gradient-variation-based detection-and-adjustment mechanism that requires no prior knowledge of the Hölder exponent or smoothness constant; and integrates a guess-and-verify framework with online-to-batch conversion to uniformly handle both smooth and nonsmooth convex functions. Theoretically, our method achieves the optimal regret bound in online learning. For offline strongly convex optimization, it is the first to attain accelerated convergence—$O(1/k^2)$—without assuming known smoothness, while retaining near-optimal $O(1/k)$ convergence in the nonsmooth case. The algorithm thus offers both universality across function classes and robustness to unknown problem parameters.

2 citationsRead paper

Recent publications

Latest Papers

A Tight Scale-Locality Bound for Partial Detection in Non-Adaptive Group Testing

Aug 12, 2026

This work addresses the fundamental challenge in non-adaptive group testing of efficiently identifying any ℓ defective items when the total number of defectives is unknown. Through information-theoretic analysis and a multi-scale testing strategy, the paper establishes the first tight lower bound on the sample complexity for this partial recovery task and provides a matching upper bound construction. The core contribution lies in proving that the optimal sample complexity of randomized non-adaptive algorithms in this setting is Θ(ℓ log²(n/ℓ)), with precisely matching upper and lower bounds. This result fully characterizes the theoretical limits of the problem and reveals an intrinsic trade-off between test size and locality.

0 citationsRead paper

An FKN Theorem for the Binary Grassmann Scheme

Aug 11, 2026

This work extends the classical Friedgut–Kalai–Naor (FKN) theorem to the binary Grassmann scheme, investigating the structural properties of Boolean functions defined on the set of ℓ-dimensional subspaces that are close to degree-one functions. By combining combinatorial arguments, Fourier analysis, and finite field geometry, the authors establish the first FKN-type structure theorem in this setting, showing that any such Boolean function—or its complement—must be close to a simple function determined by a finite collection of point and hyperplane indicators. This result reveals an intrinsic rigidity of approximately linear Boolean functions over high-dimensional subspaces and significantly broadens the understanding of structural properties of Boolean functions beyond classical domains.

0 citationsRead paper

Estimating the perturbed utility route choice model with trip-level data

Aug 11, 2026

This study addresses the challenge of estimating perturbed-utility route choice models using individual-level travel data by proposing a novel nested fixed-point algorithm. The approach employs bias-corrected linear regression at the upper level and solves individual perturbed utility maximization problems at the lower level. This method constitutes the first identifiable estimation framework for such microeconomic models and establishes a corresponding theory for statistical inference. Simulation experiments confirm the consistency and asymptotic normality of the proposed estimator, while its effectiveness and practical applicability are demonstrated through successful implementation on large-scale real-world travel data.

0 citationsRead paper

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

Aug 11, 2026

This work addresses the prevalent issue of hallucinations in large vision-language models (LVLMs), where generated text often contradicts visual content, necessitating fine-grained intervention. The authors propose UniProbe, a lightweight, learnable token-level hallucination detector that unifies heterogeneous internal representations—image patches, query tokens, and generated tokens—into a directed relational graph for the first time. By integrating spatial, sequential, and relational information through an alternating structure-aware module, UniProbe enables real-time hallucination detection and resampling during streaming decoding with only a single forward pass. Evaluated across multiple LVLM backbones, UniProbe achieves state-of-the-art performance in both token-level and object-level hallucination detection, reducing object hallucinations by up to 55% during decoding while introducing merely 6% additional latency.

0 citationsRead paper

Decoding-Level Taboo: A Diagnostic Stress Test for LLM Robustness

Aug 10, 2026

This work addresses the performance degradation of large language models (LLMs) in real-world deployment, where system prompts or safety constraints often divert generation from standard pathways despite strong benchmark results. To diagnose such robustness gaps, the authors propose a prompt-free stress-testing methodology that dynamically intervenes in the logit space during decoding by masking dominant candidate tokens at word boundaries, thereby compelling the model to generate via alternative, non-canonical paths. This approach introduces a novel runtime dynamic token-masking mechanism, establishing a zero-prompt stress-testing paradigm. Experiments across multiple open-source LLMs reveal significant effects of model scale and instruction alignment on robustness under such perturbations. Furthermore, the method efficiently produces diverse synthetic data, enabling pre-deployment reliability auditing and evaluation of safety mechanisms.

0 citationsRead paper