Institution profile

Northeastern University

Academic institutionnorthamerica · us
Official website
Research library2,243linked papers
Opportunities0open roles
Selected work

Representative Papers

Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Apr 10, 2025Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning

Large language models (LLMs) suffer from low data efficiency, typically requiring trillion-word corpora for effective pretraining. Method: Inspired by child language acquisition, this work proposes a cognitively grounded, highly efficient pretraining paradigm using only a developmentally appropriate corpus of under 100 million tokens. We systematically demonstrate—contrary to prevailing assumptions—that such small-scale data can surpass trillion-parameter models’ performance when combined with short-sequence training, knowledge distillation, and multi-task evaluation (covering syntactic competence, downstream task transfer, and out-of-distribution generalization); notably, curriculum learning proves ineffective in this low-data regime. Contribution/Results: Leveraging the LTG-BERT architecture, our best-performing model achieves state-of-the-art results across diverse benchmarks, significantly outperforming standard large baselines. The project yields over 30 empirically validated guidelines—identifying both viable strategies and dead ends—for efficient pretraining, thereby establishing a novel paradigm for cognitive modeling and environmentally sustainable (“green”) AI.

105 citations18 influentialRead paper

Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores

Mar 01, 2024arXiv.org

A lack of standardized, reproducible methods for quantifying textual diversity in large language models (LLMs) hinders rigorous evaluation of generation quality and cross-model or cross-corpus comparisons. Method: We propose the first systematic framework for text diversity evaluation, empirically validating convergent validity of diversity metrics and identifying a minimal, complete metric set—comprising compression ratio (zlib/lz4), long n-gram self-repetition rate, Self-BLEU, and BERTScore—that exhibits low inter-metric correlation and complementary multidimensional coverage. Contribution/Results: We release *diversity*, an open-source Python library enabling efficient computation and interactive visualization. Empirical analysis demonstrates that lightweight compression-based metrics robustly substitute for computationally expensive n-gram homogeneity scores. The framework substantially enhances interpretability, comparability, and practical utility of diversity assessment in LLM research.

26 citations2 influentialRead paper

Rethinking Membership Inference Attacks Against Transfer Learning

Jan 20, 2025IEEE Transactions on Information Forensics and Security

In transfer learning, teacher models’ training data are vulnerable to membership inference attacks (MIAs), yet existing work predominantly assumes attackers have black-box or white-box access to the teacher model—overlooking privacy threats when only the student model is accessible in a white-box setting. Method: This paper identifies that representational discrepancies between teacher and student hidden layers can be exploited for MIAs, and proposes the first MIA framework requiring only white-box access to the student model. It employs shadow models to calibrate student-layer representations, models teacher–student representation divergence, and infers teacher training membership via reverse inference—without requiring the teacher model itself. Contribution/Results: The method achieves high inference accuracy across four benchmark datasets and diverse transfer learning tasks. It significantly expands the threat surface of MIAs in transfer learning and provides novel empirical evidence and insights for safeguarding teacher model privacy.

13 citationsRead paper

Attacking the Spike: On the Transferability and Security of Spiking Neural Networks to Adversarial Examples

Sep 07, 2022

This work systematically investigates the underexplored problem of adversarial robustness in Spiking Neural Networks (SNNs). We find that white-box attacks against SNNs heavily rely on surrogate gradient techniques and exhibit severely limited transferability of adversarial examples across architectures (e.g., between SNNs and ViTs/CNNs). To address this, we first uncover a strong coupling mechanism between SNNs’ adversarial vulnerability and surrogate gradient estimation. Building on this insight, we propose Auto-SAGA—a cross-architecture universal white-box attack method that jointly optimizes adaptive self-attention gradient estimation and surrogate gradient approximation. Evaluated on CIFAR-10, CIFAR-100, and ImageNet, Auto-SAGA achieves a 91.1% improvement in attack success rate on SNN-ViT ensembles and attains three times the effectiveness of Auto-PGD on adversarially trained SNN ensembles, significantly outperforming existing baselines.

13 citationsRead paper

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

Jan 17, 2026

This work addresses the challenge that existing AI agent benchmarks inadequately evaluate performance on real-world, complex, and long-horizon command-line tasks. To bridge this gap, the authors introduce a novel evaluation benchmark comprising 89 high-difficulty terminal tasks, all derived from authentic workflows and accompanied by isolated execution environments, human-authored reference solutions, and automated verification tests. The benchmark is designed to ensure realism, verifiability, and diversity, substantially narrowing the disparity between practical scenarios and current model evaluation paradigms. Experimental results demonstrate that even state-of-the-art agents achieve success rates below 65% on this benchmark. The paper further provides comprehensive error analysis and publicly releases the dataset and evaluation toolchain to support future research in this domain.

9 citations1 influentialRead paper
Recent publications

Latest Papers