Triangle Multiplication Is All You Need For Biomolecular Structure Representations

2025-10-21

AI Summary

AlphaFold3’s Pairformer employs computationally expensive triangular attention, limiting scalability for large-scale biomolecular structure prediction. This work introduces Pairmixer, the first architecture to eliminate triangular attention entirely while preserving high-order geometric reasoning capability—replacing it with a lightweight triangular multiplicative update mechanism for efficient pairwise representation modeling. This design significantly reduces the computational complexity of long-range dependency modeling and enhances sequence scalability. Experiments demonstrate state-of-the-art performance on protein folding and molecular docking tasks; inference speed improves by 4× and training cost decreases by 34%; BoltzDesign sampling accelerates by over 2×, enabling longer-sequence modeling. Pairmixer provides an efficient, scalable foundation for large-scale applications including virtual screening, whole-proteome folding, and de novo binder design.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

AlphaFold has transformed protein structure prediction, but emerging applications such as virtual ligand screening, proteome-wide folding, and de novo binder design demand predictions at a massive scale, where runtime and memory costs become prohibitive. A major bottleneck lies in the Pairformer backbone of AlphaFold3-style models, which relies on computationally expensive triangular primitives-especially triangle attention-for pairwise reasoning. We introduce Pairmixer, a streamlined alternative that eliminates triangle attention while preserving higher-order geometric reasoning capabilities that are critical for structure prediction. Pairmixer substantially improves computational efficiency, matching state-of-the-art structure predictors across folding and docking benchmarks, delivering up to 4x faster inference on long sequences while reducing training cost by 34%. Its efficiency alleviates the computational burden of downstream applications such as modeling large protein complexes, high-throughput ligand and binder screening, and hallucination-based design. Within BoltzDesign, for example, Pairmixer delivers over 2x faster sampling and scales to sequences ~30% longer than the memory limits of Pairformer.

Problem

• Efficient biomolecular structure prediction at massive scale
• Reducing computational costs of triangular attention mechanisms
• Enabling large-scale protein complex modeling and screening

Innovation

• Replaces triangle attention with streamlined Pairmixer
• Preserves geometric reasoning for structure prediction
• Improves computational efficiency and reduces training cost