Multimodal deep learning from spectra for small-molecule structure identification: enhancing robustness with mixed-condition training
研究通过混合条件训练和多专家融合方法，提高小分子结构识别的鲁棒性，解决光谱缺失、降解或不匹配的问题。
0 citationsRead paper
Selected work
Representative Papers
URNet: A Unified Reparameterized Network for Efficient RGB-D Semantic Segmentation
This work addresses the limitations of existing RGB-D semantic segmentation methods, which typically employ dual-encoder architectures that suffer from inadequate depth representation, restricted cross-modal interaction, and computational redundancy. To overcome these issues, we propose URNet, a unified framework that processes both RGB and depth inputs through a single encoder to enable efficient multi-scale feature fusion. The core innovations include integrating reparameterized blocks (RepBlocks) with Linear Gated Attention (LGA) modules, allowing simultaneous feature extraction and cross-modal interaction within a unified architecture, as well as designing a lightweight, general-purpose Pyramid Merging Decoder (PMD). Extensive experiments demonstrate that URNet achieves state-of-the-art performance across multiple benchmarks while significantly improving inference efficiency.
0 citationsRead paper
Guideline-as-Oracle: Zero-Annotation Training of an Ophthalmic Telephone Triage Agent
This work addresses the scarcity of supervised signals in multi-turn ophthalmic telephone triage, caused by high expert annotation costs and clinical privacy constraints. To overcome this challenge, the authors propose the Guideline-as-Oracle (GAO) framework, which translates the American Academy of Ophthalmology guidelines into 70 operational rules to serve as the sole instance-level supervision for 3,000 training dialogues—eliminating the need for manual annotation. By introducing eight novel rule-to-dialogue construction strategies, a rule-based zero-annotation dialogue generation method, a label repair mechanism, and fine-tuning a 9B-parameter language model, GAO-Triage achieves a significant improvement on a reference set of 201 cases: expert agreement rises from 61.7% to 74.1%, and recall for urgent cases increases dramatically from 9.5% to 69.0%. The system outperforms all general-purpose baselines without relying on advanced reasoning techniques.
0 citationsRead paper
RSGMamba: Reliability-Aware Self-Gated State Space Model for Multimodal Semantic Segmentation
This work addresses the limitation of existing cross-modal fusion methods, which typically assume all modalities are equally reliable and consequently suffer significant performance degradation when auxiliary modalities contain noise, misalignment, or missing data. To overcome this, the authors propose a reliability-aware, self-gated State Space Model (Mamba) that dynamically selects and aggregates trustworthy features through a self-gating mechanism. Additionally, a lightweight local cross-gating modulation is introduced to enhance fine-grained detail modeling, enabling efficient fusion of both global and local multimodal features. The proposed method achieves state-of-the-art performance across multiple benchmarks—including NYUDepth V2, SUN-RGBD, MFNet, and PST900—with a maximum mIoU improvement of 1.6% while using only 48.6 million parameters.
0 citationsRead paper
Brain4FMs: A Benchmark of Foundation Models for Electrical Brain Signal
This study addresses the lack of a unified evaluation framework and systematic understanding of foundation models for brain signals. To this end, we present Brain4FMs, the first benchmarking platform specifically designed for neural signal foundation models such as EEG and intracranial EEG. The platform introduces a modular, open, and standardized architecture enabling fair cross-model and cross-task comparisons. It integrates 15 representative self-supervised foundation models and 18 public datasets spanning multiple neural signal modalities. We systematically evaluate how pretraining data, self-supervised learning strategies, and model architectures influence generalization performance. This work establishes the first taxonomy and unified evaluation protocol for brain signal foundation models, thereby advancing the development of more accurate and transferable neural signal modeling approaches.
0 citationsRead paper
Recent publications
Latest Papers
Multimodal deep learning from spectra for small-molecule structure identification: enhancing robustness with mixed-condition training
研究通过混合条件训练和多专家融合方法，提高小分子结构识别的鲁棒性，解决光谱缺失、降解或不匹配的问题。
0 citationsRead paper
URNet: A Unified Reparameterized Network for Efficient RGB-D Semantic Segmentation
This work addresses the limitations of existing RGB-D semantic segmentation methods, which typically employ dual-encoder architectures that suffer from inadequate depth representation, restricted cross-modal interaction, and computational redundancy. To overcome these issues, we propose URNet, a unified framework that processes both RGB and depth inputs through a single encoder to enable efficient multi-scale feature fusion. The core innovations include integrating reparameterized blocks (RepBlocks) with Linear Gated Attention (LGA) modules, allowing simultaneous feature extraction and cross-modal interaction within a unified architecture, as well as designing a lightweight, general-purpose Pyramid Merging Decoder (PMD). Extensive experiments demonstrate that URNet achieves state-of-the-art performance across multiple benchmarks while significantly improving inference efficiency.
0 citationsRead paper
Guideline-as-Oracle: Zero-Annotation Training of an Ophthalmic Telephone Triage Agent
This work addresses the scarcity of supervised signals in multi-turn ophthalmic telephone triage, caused by high expert annotation costs and clinical privacy constraints. To overcome this challenge, the authors propose the Guideline-as-Oracle (GAO) framework, which translates the American Academy of Ophthalmology guidelines into 70 operational rules to serve as the sole instance-level supervision for 3,000 training dialogues—eliminating the need for manual annotation. By introducing eight novel rule-to-dialogue construction strategies, a rule-based zero-annotation dialogue generation method, a label repair mechanism, and fine-tuning a 9B-parameter language model, GAO-Triage achieves a significant improvement on a reference set of 201 cases: expert agreement rises from 61.7% to 74.1%, and recall for urgent cases increases dramatically from 9.5% to 69.0%. The system outperforms all general-purpose baselines without relying on advanced reasoning techniques.
0 citationsRead paper
RSGMamba: Reliability-Aware Self-Gated State Space Model for Multimodal Semantic Segmentation
This work addresses the limitation of existing cross-modal fusion methods, which typically assume all modalities are equally reliable and consequently suffer significant performance degradation when auxiliary modalities contain noise, misalignment, or missing data. To overcome this, the authors propose a reliability-aware, self-gated State Space Model (Mamba) that dynamically selects and aggregates trustworthy features through a self-gating mechanism. Additionally, a lightweight local cross-gating modulation is introduced to enhance fine-grained detail modeling, enabling efficient fusion of both global and local multimodal features. The proposed method achieves state-of-the-art performance across multiple benchmarks—including NYUDepth V2, SUN-RGBD, MFNet, and PST900—with a maximum mIoU improvement of 1.6% while using only 48.6 million parameters.
0 citationsRead paper
Brain4FMs: A Benchmark of Foundation Models for Electrical Brain Signal
This study addresses the lack of a unified evaluation framework and systematic understanding of foundation models for brain signals. To this end, we present Brain4FMs, the first benchmarking platform specifically designed for neural signal foundation models such as EEG and intracranial EEG. The platform introduces a modular, open, and standardized architecture enabling fair cross-model and cross-task comparisons. It integrates 15 representative self-supervised foundation models and 18 public datasets spanning multiple neural signal modalities. We systematically evaluate how pretraining data, self-supervised learning strategies, and model architectures influence generalization performance. This work establishes the first taxonomy and unified evaluation protocol for brain signal foundation models, thereby advancing the development of more accurate and transferable neural signal modeling approaches.
0 citationsRead paper