Institution profile

University of Notre Dame

Academic institutionnorthamerica · us
Official website
Research library788linked papers
Opportunities0open roles
Selected work

Representative Papers

MolX: Enhancing Large Language Models for Molecular Learning with A Multi-Modal Extension

Jun 10, 2024arXiv.org

Large language models (LLMs) exhibit limited molecular structural understanding—especially when relying solely on one-dimensional textual representations like SMILES—hindering their effectiveness in chemistry. Method: We propose MolX, a lightweight multimodal extension module that jointly encodes SMILES sequences, 2D molecular graphs (via GNNs), and expert-crafted molecular fingerprints. MolX is trained via multitask contrastive learning while keeping the LLM backbone frozen. Contribution/Results: MolX establishes the first “frozen-LLM + multimodal alignment” paradigm, introducing only 0.53%–0.82% additional trainable parameters. It achieves significant improvements over baselines across four downstream tasks—including molecule-to-text translation and retrosynthetic planning—while supporting both zero-shot inference and fine-tuning deployment. This enhances cross-task generalization of LLMs in chemistry without architectural modification or full-parameter adaptation.

8 citations2 influentialRead paper

Addressing Overthinking in Large Vision-Language Models via Gated Perception-Reasoning Optimization

Jan 07, 2026arXiv.org

This work addresses the inefficiency and accuracy degradation of large vision-language models (VLMs) on simple tasks, where over-reasoning often leads to unnecessarily verbose responses. While prior approaches overlook visual perception failure as a fundamental bottleneck, this paper proposes GPRO, a novel framework that decouples perception failures from reasoning errors for the first time. GPRO constructs supervision signals based on failure attribution and introduces a meta-reasoning controller that dynamically selects among a lightweight fast path, a slow perception path, or a slow reasoning path. Leveraging a teacher model to generate approximately 790,000 failure-attribution labels, the path selection strategy is optimized via multi-objective reinforcement learning. Experiments demonstrate that GPRO significantly improves both accuracy and inference efficiency across five benchmarks, outperforming existing "slow thinking" methods while producing more concise responses.

3 citationsRead paper

MoCo: A One-Stop Shop for Model Collaboration Research

Jan 29, 2026

This work addresses the lack of a unified benchmark in existing research on model collaboration, which hinders systematic evaluation of diverse collaborative strategies. To bridge this gap, we propose MoCo—a modular Python library that, for the first time, systematically integrates 26 collaboration methods, supporting multi-granularity information exchange at the levels of routing, text, logits, and parameters. MoCo incorporates 25 diverse evaluation datasets and provides an extensible framework for heterogeneous model collaboration alongside efficient analysis tools, thereby establishing model collaboration as a distinct research paradigm. Experimental results demonstrate that collaborative strategies outperform single-model baselines in 61.0% of (model, dataset) configurations, with the best-performing method achieving a performance gain of up to 25.8%, highlighting the substantial advantages of collaboration in complex tasks.

2 citationsRead paper

OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG

Jan 13, 2026

This work proposes OpenDecoder, a novel framework that addresses the challenge of inconsistent retrieval quality in Retrieval-Augmented Generation (RAG) systems, which often undermines answer accuracy. OpenDecoder is the first approach to explicitly integrate multi-dimensional document quality signals—including relevance scores, ranking positions, and query performance prediction metrics—directly into the decoding process of large language models (LLMs), enabling quality-aware generation control. The framework is highly flexible, allowing seamless incorporation of arbitrary external quality indicators and compatibility with various LLM post-training objectives. Extensive experiments across five benchmark datasets demonstrate that OpenDecoder significantly outperforms existing baselines, substantially enhancing the robustness of RAG systems against noisy contexts and improving the reliability of generated responses.

2 citationsRead paper

Elsewise: Authoring AI-Based Interactive Narrative with Possibility Space Visualization

Dec 21, 2025arXiv.org

Interactive narrative (IN) authors craft spaces of divergent narrative possibilities for players to explore, with the player's input determining which narrative possibilities they actually experience. Generative AI can enable new forms of IN by improvisationally expanding on pre-authored content in response to open-ended player input. However, this extrapolation risks widening the gap between author-envisioned and player-experienced stories, potentially limiting the strength of plot progression and the communication of the author's narrative intent. To bridge the gap, we introduce Elsewise: an authoring tool for AI-based INs that implements a novel Bundled Storyline concept to enhance author's perception and understanding of the narrative possibility space, allowing authors to explore similarities and differences between possible playthroughs of their IN in terms of open-ended, user-configurable narrative dimensions. A user study (n=12) shows that our approach improves author anticipation of player-experienced narrative, leading to more effective control and exploration of the narrative possibility spaces.

2 citationsRead paper
Recent publications

Latest Papers

Agent as Policy for Robotic Manipulation

Sep 11, 2026

本文提出了一种名为Agent as Policy (AGP)的方法，通过让通用智能体直接控制物理机器人完成任务，无需特定任务或环境的额外训练，解决了机器人操作中的泛化问题。

0 citationsRead paper