Molecular Quantum Transformer

2025-03-27

AI Summary

Classical Transformers incur prohibitive computational costs and struggle to represent quantum states when modeling molecular quantum systems. To address this, we propose Quantum-Transformer: the first architecture to embed parameterized quantum circuits into the self-attention mechanism, enabling end-to-end quantum encoding of molecular geometries and Hamiltonians. It integrates quantum embedding with a variational quantum eigensolver (VQE) in a jointly optimized framework, supporting cross-molecular pretraining and few-shot transfer. On H₂, LiH, BeH₂, and H₄, it achieves significantly lower ground-state energy prediction errors than classical Transformers. A pretrained model adapts efficiently to unseen molecules using only 1–3 samples, improving generalization efficiency by an order of magnitude. The core innovation lies in the deep integration of a quantum-enabled attention mechanism with a differentiable quantum-chemical modeling framework.

Technology Category

• Machine Learning: Quantum Machine Learning
• Search and Optimization: Learning to Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The Transformer model, renowned for its powerful attention mechanism, has achieved state-of-the-art performance in various artificial intelligence tasks but faces challenges such as high computational cost and memory usage. Researchers are exploring quantum computing to enhance the Transformer's design, though it still shows limited success with classical data. With a growing focus on leveraging quantum machine learning for quantum data, particularly in quantum chemistry, we propose the Molecular Quantum Transformer (MQT) for modeling interactions in molecular quantum systems. By utilizing quantum circuits to implement the attention mechanism on the molecular configurations, MQT can efficiently calculate ground-state energies for all configurations. Numerical demonstrations show that in calculating ground-state energies for H_2, LiH, BeH_2, and H_4, MQT outperforms the classical Transformer, highlighting the promise of quantum effects in Transformer structures. Furthermore, its pretraining capability on diverse molecular data facilitates the efficient learning of new molecules, extending its applicability to complex molecular systems with minimal additional effort. Our method offers an alternative to existing quantum algorithms for estimating ground-state energies, opening new avenues in quantum chemistry and materials science.

Problem

• Enhancing Transformer models with quantum computing for molecular systems
• Efficiently calculating ground-state energies of molecular configurations
• Improving quantum chemistry applications via quantum machine learning

Innovation

• Quantum circuits implement attention mechanism
• Efficient ground-state energy calculation
• Pretraining on diverse molecular data