Selected work

Representative Papers

Mechanisms of Object Localization in Vision-Language Models

May 19, 2026

This study addresses the unclear internal mechanisms underlying object localization in vision-language models (VLMs), which hinders their interpretability and performance improvement. It reveals, for the first time at the layer and attention head granularity, that object localization in LLaVA-1.5 and InternVL-3.5 relies on narrow computational pathways formed by a small subset of specialized attention heads—rather than internal semantic rearrangements—exhibiting a “containerized” mechanism. Through token ablation, attention knockout, and causal mediation analysis, the work demonstrates that localization and classification tasks share early visual processing but are driven by distinct sets of attention heads: in LLaVA, critical heads concentrate in early-to-mid layers, whereas in InternVL, they are distributed across mid-to-late layers.

0 citationsRead paper

Synthesizing Visual Concepts as Vision-Language Programs

Nov 24, 2025

Vision-language models (VLMs) often lack logical consistency in systematic visual reasoning. To address this, we propose Vision-Language Programs (VLP), a framework that automatically compiles VLM-generated natural language descriptions into executable neural-symbolic programs for structured perception and formal reasoning over images. VLP integrates program synthesis, neural-symbolic computation, and structured prompting to balance perceptual flexibility with logical rigor, enabling human-interpretable reasoning traces and mitigating shortcut learning. Evaluated on both synthetic and real-world benchmarks, VLP significantly outperforms direct prompting and structured prompting baselines, improving accuracy and output consistency on complex visual reasoning tasks. Notably, VLP achieves the first end-to-end generation of image-executable programs from natural language instructions, bridging high-level semantics with grounded, verifiable computation.

0 citationsRead paper

Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining

Sep 12, 2025

To address the O(N²) computational complexity of softmax attention in Transformers for long sequences, this paper proposes Multipole Semantic Attention (MuSe). MuSe preserves attention asymmetry by decoupling query and key spaces via semantic clustering, enhances centroid approximation accuracy using multipole expansion (monopole plus dipole correction), and supports both non-causal and causal attention through hierarchical block decomposition. As a plug-and-play module requiring no architectural modifications, MuSe reduces complexity to O(NCD) or O(NCD log N). Experiments demonstrate that on 8k-context sequences, MuSe achieves 3× speedup per layer with relative squared error under 20%; in end-to-end pretraining on 16k-context sequences, it reduces runtime by 12.2% while increasing loss by only 0.36%. These results significantly improve efficiency for long-sequence modeling without compromising model fidelity.

0 citationsRead paper

Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback

Aug 14, 2025

Novelty assessment in academic peer review remains a critical yet underexplored challenge. Method: This paper introduces the first large language model–based, structured novelty assessment framework. It emulates expert reviewer behavior via a three-stage automated pipeline: (1) structured extraction of submission content, (2) literature-aware retrieval and synthesis of related work, and (3) claim-level comparative reasoning—explicitly modeling independent claim verification and contextual inference. The method integrates analysis of large-scale human review corpora, literature-aware information extraction, and evidence-driven judgment techniques. Contribution/Results: Evaluated on 182 submissions to ICLR 2025, the framework achieves 86.5% alignment with human reviewers’ reasoning processes and 75.3% agreement on final novelty judgments—substantially outperforming existing baselines—while markedly improving assessment transparency and consistency.

0 citationsRead paper

Finding Dori: Memorization in Text-to-Image Diffusion Models Is Less Local Than Assumed

Jul 22, 2025

Text-to-image diffusion models pose privacy and intellectual property risks due to memorization of training data. Existing defenses rely unverifiedly on the “localization-of-memory” assumption—suppressing memorization via weight pruning—but this assumption lacks empirical validation and such methods are easily circumvented. This paper is the first to systematically demonstrate that memorization exhibits *non-locality*: a single memorized sample can be triggered by diverse text embedding positions and multiple computational pathways. Leveraging this insight, we propose a novel adversarial iterative fine-tuning paradigm that jointly performs text embedding space probing, robust trigger search, and targeted unlearning—effectively blocking memory leakage without compromising generation fidelity. Experiments across diverse benchmarks show substantial improvements in defense robustness against adaptive attacks. Our approach delivers a verifiable, deployable, and generation-preserving unlearning mechanism, advancing regulatory compliance and trustworthy deployment of generative AI.

0 citationsRead paper

Recent publications

Latest Papers

Mechanisms of Object Localization in Vision-Language Models

May 19, 2026

This study addresses the unclear internal mechanisms underlying object localization in vision-language models (VLMs), which hinders their interpretability and performance improvement. It reveals, for the first time at the layer and attention head granularity, that object localization in LLaVA-1.5 and InternVL-3.5 relies on narrow computational pathways formed by a small subset of specialized attention heads—rather than internal semantic rearrangements—exhibiting a “containerized” mechanism. Through token ablation, attention knockout, and causal mediation analysis, the work demonstrates that localization and classification tasks share early visual processing but are driven by distinct sets of attention heads: in LLaVA, critical heads concentrate in early-to-mid layers, whereas in InternVL, they are distributed across mid-to-late layers.

0 citationsRead paper

Synthesizing Visual Concepts as Vision-Language Programs

Nov 24, 2025

Vision-language models (VLMs) often lack logical consistency in systematic visual reasoning. To address this, we propose Vision-Language Programs (VLP), a framework that automatically compiles VLM-generated natural language descriptions into executable neural-symbolic programs for structured perception and formal reasoning over images. VLP integrates program synthesis, neural-symbolic computation, and structured prompting to balance perceptual flexibility with logical rigor, enabling human-interpretable reasoning traces and mitigating shortcut learning. Evaluated on both synthetic and real-world benchmarks, VLP significantly outperforms direct prompting and structured prompting baselines, improving accuracy and output consistency on complex visual reasoning tasks. Notably, VLP achieves the first end-to-end generation of image-executable programs from natural language instructions, bridging high-level semantics with grounded, verifiable computation.

0 citationsRead paper

Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining

Sep 12, 2025

To address the O(N²) computational complexity of softmax attention in Transformers for long sequences, this paper proposes Multipole Semantic Attention (MuSe). MuSe preserves attention asymmetry by decoupling query and key spaces via semantic clustering, enhances centroid approximation accuracy using multipole expansion (monopole plus dipole correction), and supports both non-causal and causal attention through hierarchical block decomposition. As a plug-and-play module requiring no architectural modifications, MuSe reduces complexity to O(NCD) or O(NCD log N). Experiments demonstrate that on 8k-context sequences, MuSe achieves 3× speedup per layer with relative squared error under 20%; in end-to-end pretraining on 16k-context sequences, it reduces runtime by 12.2% while increasing loss by only 0.36%. These results significantly improve efficiency for long-sequence modeling without compromising model fidelity.

0 citationsRead paper

Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback

Aug 14, 2025

Novelty assessment in academic peer review remains a critical yet underexplored challenge. Method: This paper introduces the first large language model–based, structured novelty assessment framework. It emulates expert reviewer behavior via a three-stage automated pipeline: (1) structured extraction of submission content, (2) literature-aware retrieval and synthesis of related work, and (3) claim-level comparative reasoning—explicitly modeling independent claim verification and contextual inference. The method integrates analysis of large-scale human review corpora, literature-aware information extraction, and evidence-driven judgment techniques. Contribution/Results: Evaluated on 182 submissions to ICLR 2025, the framework achieves 86.5% alignment with human reviewers’ reasoning processes and 75.3% agreement on final novelty judgments—substantially outperforming existing baselines—while markedly improving assessment transparency and consistency.

0 citationsRead paper

Finding Dori: Memorization in Text-to-Image Diffusion Models Is Less Local Than Assumed

Jul 22, 2025

Text-to-image diffusion models pose privacy and intellectual property risks due to memorization of training data. Existing defenses rely unverifiedly on the “localization-of-memory” assumption—suppressing memorization via weight pruning—but this assumption lacks empirical validation and such methods are easily circumvented. This paper is the first to systematically demonstrate that memorization exhibits *non-locality*: a single memorized sample can be triggered by diverse text embedding positions and multiple computational pathways. Leveraging this insight, we propose a novel adversarial iterative fine-tuning paradigm that jointly performs text embedding space probing, robust trigger search, and targeted unlearning—effectively blocking memory leakage without compromising generation fidelity. Experiments across diverse benchmarks show substantial improvements in defense robustness against adaptive attacks. Our approach delivers a verifiable, deployable, and generation-preserving unlearning mechanism, advancing regulatory compliance and trustworthy deployment of generative AI.

0 citationsRead paper