Exploring the Alignment of Generation and Understanding in Protein Structure Modeling

2026-07-15

AI Summary

Current protein generative models exhibit suboptimal performance on understanding tasks, revealing a representational gap between generation and comprehension capabilities. To address this issue, this work introduces Representation Alignment (REPA)—a novel framework adapted to the protein domain—that explicitly aligns the representation spaces of diffusion-based generative models with those of pretrained understanding encoders during training. By bridging the semantic discrepancy between these two modalities, REPA substantially enhances the model’s comprehension abilities. Empirical evaluation on the MotifBench benchmark demonstrates that integrating REPA into Protpardelle-1c improves its performance from 39.2 to 47.1, yielding a 20% relative gain and confirming the efficacy of representation alignment in unifying generative and understanding tasks in protein modeling.

Abstract

Understanding and generation are often treated as two separate paradigms in training deep neural networks, despite the fact that both are trained with closely related objectives such as denoising and masked prediction. While prior studies have shown that generative models often learn suboptimal representations for understanding tasks in vision, it is less understood whether a similar gap exists in the protein domain. In this work, we systematically investigate this question by benchmarking state-of-the-art protein generative models on widely-used protein understanding tasks, and observe that these models exhibit consistently poor performance compared to existing protein encoders. Furthermore, inspired by the Representation Alignment (REPA) framework, we propose to explicitly align generative protein diffusion models with pretrained protein understanding models during training. Experiments on the MotifBench demonstrate that representation alignment significantly improves functional protein generation, boosting the MotifBench score of Protpardelle-1c from 39.2 to 47.1, corresponding to a 20% relative improvement. Our results suggest that representation alignment provides a general and effective mechanism for bridging understanding and generation in protein structure modeling.

Problem

• protein structure modeling
• generative models
• understanding tasks
• representation gap
• alignment

Innovation

• representation alignment
• protein generative modeling
• diffusion models
• protein understanding
• MotifBench