Selected work

Representative Papers

Characterizing Mamba's Selective Memory using Auto-Encoders

Dec 17, 2025

This work investigates the selective forgetting mechanism of state space models (SSMs)—particularly the Mamba family (130M–1.4B parameters)—under fixed memory constraints when processing long sequences. To address *which semantic types and sequences are more prone to forgetting*, we propose an autoencoder-based latent-state reconstruction evaluation framework that quantifies information loss across token categories (e.g., part-of-speech tags, named entities) and sequence domains (e.g., code, mathematical problems). Our systematic analysis reveals, for the first time, that low-frequency tokens—including mathematical symbols, organizational named entities, and non-standard American English—exhibit significantly higher forgetting rates; crucially, forgetting magnitude is strongly negatively correlated with token frequency in the pretraining corpus. This establishes an interpretable, data-distribution-aware linkage between forgetting patterns and training statistics, providing both empirical grounding and a diagnostic tool for memory modeling and long-context optimization in SSMs.

0 citationsRead paper

Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions

Sep 02, 2025

This study investigates whether large language models (LLMs) prioritize externally provided explicit label definitions or rely predominantly on internal parametric knowledge during reasoning. To address this, we conduct controlled experiments across general-purpose benchmarks (e.g., BoolQ, MultiRC) and domain-specific benchmarks (e.g., MedQA, SciTail), systematically evaluating model adherence to human-annotated versus LLM-generated label definitions. Results reveal that external definition integration is neither robust nor consistent: in general tasks, models heavily default to internal representations, whereas in domain-specific tasks, external definitions improve accuracy (by +2.1–5.7%) and enhance decision interpretability. We identify, for the first time, a “task-sensitivity” phenomenon in definition adoption, propose a novel quantitative metric for measuring definition adherence, and underscore the critical importance of modeling knowledge fusion mechanisms to achieve controllable, reliable reasoning.

0 citationsRead paper

Operationalizing AI for Good: Spotlight on Deployment and Integration of AI Models in Humanitarian Work

Jul 21, 2025

Existing AI for Good research predominantly focuses on model development, overlooking practical challenges—such as collaborative deployment with humanitarian organizations, sustained operations, and performance maintenance—in resource-constrained environments. This paper presents the first systematic investigation of AI deployment in real-world humanitarian settings, particularly those characterized by low bandwidth and high operational volatility. Methodologically, we propose a deployment framework balancing technical feasibility and organizational adaptability, integrating lightweight models, edge computing, continual learning, and modular API design to enable localized iteration and long-term operational sustainability. Our contributions are threefold: (1) a reusable, co-designed deployment paradigm; (2) empirical validation of long-term model stability and effectiveness in authentic field conditions; and (3) a practitioner-oriented technical adaptation guide for humanitarian organizations, demonstrably enhancing frontline decision-making efficiency and AI’s sustainable impact.

0 citationsRead paper

Context-Aware Regularization with Markovian Integration for Attention-Based Nucleotide Analysis

Jul 12, 2025

To address the challenges of capturing long-range dependencies and enforcing global transition consistency in modeling long nucleic acid sequences with Transformers, this paper proposes CARMANIA. The framework introduces a context-aware Markov regularization mechanism, explicitly constraining global consistency of sequence state transitions via an n-gram statistics-guided transition matrix (TM) loss—thereby jointly preserving local contextual information and evolutionary/functional structural patterns. CARMANIA employs self-supervised pretraining within a fixed window, jointly optimizing standard self-attention and the TM loss. Evaluated on 40 genomic tasks, it achieves statistically significant accuracy improvements on 33 tasks, outperforming the best long-context baseline by ≥7 percentage points on average. Notably, enhancer prediction sees a maximum MCC improvement of 34 percentage points. Moreover, inference is accelerated by 2.5× compared to the baseline.

0 citationsRead paper

NLP for Social Good: A Survey of Challenges, Opportunities, and Responsible Deployment

May 28, 2025

This paper addresses the lack of systematic responsible governance in applying NLP technologies to urgent societal challenges—including educational equity, public health, and disaster response—by proposing the NLP4SG (NLP for Social Good) research paradigm. Methodologically, it introduces the first multidimensional evaluation framework, defining a “responsibility readiness” metric system that integrates technical auditing, ethical impact assessment, participatory design, and explainability analysis, while coupling LLM capability mapping with societal need alignment. Key contributions include: (1) identifying 12 high-priority societal application scenarios; (2) uncovering seven recurrent risks—including data colonialism and feedback-loop bias; and (3) proposing 15 actionable, responsibility-oriented deployment guidelines. The work establishes a theoretical anchor, an operational assessment toolkit, and concrete implementation pathways for socially beneficial NLP development and deployment.

0 citationsRead paper

Recent publications

Latest Papers

Characterizing Mamba's Selective Memory using Auto-Encoders

Dec 17, 2025

This work investigates the selective forgetting mechanism of state space models (SSMs)—particularly the Mamba family (130M–1.4B parameters)—under fixed memory constraints when processing long sequences. To address *which semantic types and sequences are more prone to forgetting*, we propose an autoencoder-based latent-state reconstruction evaluation framework that quantifies information loss across token categories (e.g., part-of-speech tags, named entities) and sequence domains (e.g., code, mathematical problems). Our systematic analysis reveals, for the first time, that low-frequency tokens—including mathematical symbols, organizational named entities, and non-standard American English—exhibit significantly higher forgetting rates; crucially, forgetting magnitude is strongly negatively correlated with token frequency in the pretraining corpus. This establishes an interpretable, data-distribution-aware linkage between forgetting patterns and training statistics, providing both empirical grounding and a diagnostic tool for memory modeling and long-context optimization in SSMs.

0 citationsRead paper

Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions

Sep 02, 2025

This study investigates whether large language models (LLMs) prioritize externally provided explicit label definitions or rely predominantly on internal parametric knowledge during reasoning. To address this, we conduct controlled experiments across general-purpose benchmarks (e.g., BoolQ, MultiRC) and domain-specific benchmarks (e.g., MedQA, SciTail), systematically evaluating model adherence to human-annotated versus LLM-generated label definitions. Results reveal that external definition integration is neither robust nor consistent: in general tasks, models heavily default to internal representations, whereas in domain-specific tasks, external definitions improve accuracy (by +2.1–5.7%) and enhance decision interpretability. We identify, for the first time, a “task-sensitivity” phenomenon in definition adoption, propose a novel quantitative metric for measuring definition adherence, and underscore the critical importance of modeling knowledge fusion mechanisms to achieve controllable, reliable reasoning.

0 citationsRead paper

Operationalizing AI for Good: Spotlight on Deployment and Integration of AI Models in Humanitarian Work

Jul 21, 2025

Existing AI for Good research predominantly focuses on model development, overlooking practical challenges—such as collaborative deployment with humanitarian organizations, sustained operations, and performance maintenance—in resource-constrained environments. This paper presents the first systematic investigation of AI deployment in real-world humanitarian settings, particularly those characterized by low bandwidth and high operational volatility. Methodologically, we propose a deployment framework balancing technical feasibility and organizational adaptability, integrating lightweight models, edge computing, continual learning, and modular API design to enable localized iteration and long-term operational sustainability. Our contributions are threefold: (1) a reusable, co-designed deployment paradigm; (2) empirical validation of long-term model stability and effectiveness in authentic field conditions; and (3) a practitioner-oriented technical adaptation guide for humanitarian organizations, demonstrably enhancing frontline decision-making efficiency and AI’s sustainable impact.

0 citationsRead paper

Context-Aware Regularization with Markovian Integration for Attention-Based Nucleotide Analysis

Jul 12, 2025

To address the challenges of capturing long-range dependencies and enforcing global transition consistency in modeling long nucleic acid sequences with Transformers, this paper proposes CARMANIA. The framework introduces a context-aware Markov regularization mechanism, explicitly constraining global consistency of sequence state transitions via an n-gram statistics-guided transition matrix (TM) loss—thereby jointly preserving local contextual information and evolutionary/functional structural patterns. CARMANIA employs self-supervised pretraining within a fixed window, jointly optimizing standard self-attention and the TM loss. Evaluated on 40 genomic tasks, it achieves statistically significant accuracy improvements on 33 tasks, outperforming the best long-context baseline by ≥7 percentage points on average. Notably, enhancer prediction sees a maximum MCC improvement of 34 percentage points. Moreover, inference is accelerated by 2.5× compared to the baseline.

0 citationsRead paper

NLP for Social Good: A Survey of Challenges, Opportunities, and Responsible Deployment

May 28, 2025

This paper addresses the lack of systematic responsible governance in applying NLP technologies to urgent societal challenges—including educational equity, public health, and disaster response—by proposing the NLP4SG (NLP for Social Good) research paradigm. Methodologically, it introduces the first multidimensional evaluation framework, defining a “responsibility readiness” metric system that integrates technical auditing, ethical impact assessment, participatory design, and explainability analysis, while coupling LLM capability mapping with societal need alignment. Key contributions include: (1) identifying 12 high-priority societal application scenarios; (2) uncovering seven recurrent risks—including data colonialism and feedback-loop bias; and (3) proposing 15 actionable, responsibility-oriented deployment guidelines. The work establishes a theoretical anchor, an operational assessment toolkit, and concrete implementation pathways for socially beneficial NLP development and deployment.

0 citationsRead paper