Leveraging Multi-modal Representations to Predict Protein Melting Temperatures

2024-12-05

AI Summary

Accurately predicting the change in protein melting temperature (ΔTm) induced by single-point mutations is critical for assessing and engineering protein thermostability. This work systematically investigates the utility of multimodal protein representations for ΔTm prediction, conducting the first comparative evaluation of sequence and structure embeddings from ESM-2, ESM-3, and AlphaFold, and proposing a regression model that fuses these heterogeneous representations. Key contributions include: (1) demonstrating that ESM-3 embeddings substantially outperform those of ESM-2 and AlphaFold on ΔTm prediction; and (2) achieving a Pearson correlation coefficient (PCC) of 0.50 on the s571 benchmark—setting a new state-of-the-art. These results underscore the pivotal role of high-fidelity language-model-derived representations in thermodynamic stability modeling and establish a novel paradigm for deep learning–driven rational protein design.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Accurately predicting protein melting temperature changes (Delta Tm) is fundamental for assessing protein stability and guiding protein engineering. Leveraging multi-modal protein representations has shown great promise in capturing the complex relationships among protein sequences, structures, and functions. In this study, we develop models based on powerful protein language models, including ESM-2, ESM-3 and AlphaFold, using various feature extraction methods to enhance prediction accuracy. By utilizing the ESM-3 model, we achieve a new state-of-the-art performance on the s571 test dataset, obtaining a Pearson correlation coefficient (PCC) of 0.50. Furthermore, we conduct a fair evaluation to compare the performance of different protein language models in the Delta Tm prediction task. Our results demonstrate that integrating multi-modal protein representations could advance the prediction of protein melting temperatures.

Problem

• Predict protein melting temperature changes accurately
• Integrate multi-modal protein representations for stability
• Compare protein language models for Delta Tm prediction

Innovation

• Utilizing multi-modal protein representations
• Employing ESM-3 for state-of-the-art performance
• Integrating protein language models effectively