Protenix-Mini+: efficient structure prediction model with scalable pairformer

2025-10-13

AI Summary

Biological macromolecular structure prediction models suffer from high inference latency and cubic time complexity—O(L³)—due to non-scalable operations such as self-attention, severely limiting their applicability to large-scale complexes. To address this, we propose PairFormer, a lightweight architecture featuring three key innovations: (1) compression of cubic-order non-scalable operations; (2) elimination of redundant structural components across modules to reduce computational overhead; and (3) design of a low-step-count atomic diffusion sampler. Evaluated on low-homology single-chain proteins, PairFormer achieves only a ~3% drop in LDDT score while accelerating inference by over 90%, striking a strong balance between accuracy and efficiency. Our approach delivers a scalable, deployable solution for large-scale biomolecular structure prediction, enabling practical application to complex systems previously intractable with standard attention-based models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Lightweight inference is critical for biomolecular structure prediction and downstream tasks, enabling efficient real-world deployment and inference-time scaling for large-scale applications. While AF3 and its variants (e.g., Protenix, Chai-1) have advanced structure prediction results, they suffer from critical limitations: high inference latency and cubic time complexity with respect to token count, both of which restrict scalability for large biomolecular complexes. To address the core challenge of balancing model efficiency and prediction accuracy, we introduce three key innovations: (1) compressing non-scalable operations to mitigate cubic time complexity, (2) removing redundant blocks across modules to reduce unnecessary overhead, and (3) adopting a few-step sampler for the atom diffusion module to accelerate inference. Building on these design principles, we develop Protenix-Mini+, a highly lightweight and scalable variant of the Protenix model. Within an acceptable range of performance degradation, it substantially improves computational efficiency. For example, in the case of low-homology single-chain proteins, Protenix-Mini+ experiences an intra-protein LDDT drop of approximately 3% relative to the full Protenix model -- an acceptable performance trade-off given its substantially 90%+ improved computational efficiency.

Problem

• Reducing high inference latency in structure prediction models
• Addressing cubic time complexity scaling with token count
• Balancing computational efficiency with prediction accuracy trade-offs

Innovation

• Compressing non-scalable operations to reduce complexity
• Removing redundant blocks across modules to cut overhead
• Adopting few-step sampler for atom diffusion acceleration