MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding

2026-07-02

AI Summary

Existing molecular vision-language models exhibit limitations in structural alignment and topological modeling, hindering their ability to accurately interpret the chemical structure and semantics of molecular images. To address this, this work proposes MolSight, a novel framework that integrates molecular topological embeddings with a vision–language alignment mechanism. MolSight introduces bond adjacency information through a graph-aware visual encoder and incorporates a molecular grounding module to precisely align visual features with chemical symbol semantics. Experimental results demonstrate that MolSight significantly outperforms current vision-language models, molecular large language models, and specialized cheminformatics tools across multiple chemical visual understanding tasks, showcasing superior reasoning capabilities for molecular image interpretation.

Abstract

Using molecular large language models (LLMs) as a unified framework for understanding molecular structures and functions is emerging as a new trend in tasks such as molecular design and drug discovery. However, these models struggle to fully capture the visual representation of molecular structures, limiting their potential. While existing molecular vision-language models (VLMs) show promise, they still face challenges in structural alignment and lack the necessary topological modeling for accurate molecular understanding. To address this, we propose MolSight, a graph-aware vision-language model framework designed to enhance the understanding of molecular images by VLMs. MolSight integrates a Molecular Topology Module to inject chemical-bond adjacency information into vision tokens, and a Molecular Grounding Module to align visual features with chemical symbolic semantics. Our experiments demonstrate that MolSight significantly outperforms existing VLMs, molecular LLMs, and specialized tools across multiple chemical visual understanding tasks, achieving a new level of molecular image reasoning.

Problem

• molecular vision-language models
• structural alignment
• topological modeling
• molecular image understanding
• visual representation

Innovation

• graph-aware
• vision-language model
• molecular topology
• chemical image understanding
• molecular grounding