Protenix-Mini: Efficient Structure Predictor via Compact Architecture, Few-Step Diffusion and Switchable pLM

2025-07-15

AI Summary

To address the challenge of balancing efficiency and accuracy in protein structure prediction under compute-constrained settings, this work proposes a lightweight end-to-end framework. Methodologically, it introduces: (i) a compact PairFormer architecture with pruned redundant Transformer modules; (ii) a novel two-step ordinary differential equation (ODE) sampling scheme replacing conventional multi-step diffusion; and (iii) a swappable protein language model (pLM), directly substituting ESM for time-consuming multiple sequence alignment (MSA) preprocessing. The entire framework is trained end-to-end under a lightweight optimization strategy. On standard benchmarks, the model achieves 40–60% reductions in parameter count and inference latency, while suffering only a marginal 1–5% drop in TM-score. This demonstrates a near-optimal trade-off between computational efficiency and structural fidelity, enabling high-accuracy, resource-efficient protein structure prediction.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Lightweight inference is critical for biomolecular structure prediction and other downstream tasks, enabling efficient real-world deployment and inference-time scaling for large-scale applications. In this work, we address the challenge of balancing model efficiency and prediction accuracy by making several key modifications, 1) Multi-step AF3 sampler is replaced by a few-step ODE sampler, significantly reducing computational overhead for the diffusion module part during inference; 2) In the open-source Protenix framework, a subset of pairformer or diffusion transformer blocks doesn't make contributions to the final structure prediction, presenting opportunities for architectural pruning and lightweight redesign; 3) A model incorporating an ESM module is trained to substitute the conventional MSA module, reducing MSA preprocessing time. Building on these key insights, we present Protenix-Mini, a compact and optimized model designed for efficient protein structure prediction. This streamlined version incorporates a more efficient architectural design with a two-step Ordinary Differential Equation (ODE) sampling strategy. By eliminating redundant Transformer components and refining the sampling process, Protenix-Mini significantly reduces model complexity with slight accuracy drop. Evaluations on benchmark datasets demonstrate that it achieves high-fidelity predictions, with only a negligible 1 to 5 percent decrease in performance on benchmark datasets compared to its full-scale counterpart. This makes Protenix-Mini an ideal choice for applications where computational resources are limited but accurate structure prediction remains crucial.

Problem

• Balancing model efficiency and prediction accuracy in protein structure prediction.
• Reducing computational overhead in diffusion module via few-step ODE sampler.
• Minimizing MSA preprocessing time by substituting with an ESM module.

Innovation

• Few-step ODE sampler reduces computational overhead
• Architectural pruning for lightweight redesign
• ESM module replaces MSA to cut preprocessing