ThermoRL:Structure-Aware Reinforcement Learning for Protein Mutation Design to Enhance Thermostability

2025-07-24

AI Summary

To address the limited search space and difficulty in discovering novel beneficial mutations in protein thermal stability design, this work proposes a structure-aware hierarchical reinforcement learning (RL) framework. Methodologically, it integrates a pre-trained graph neural network (GNN) for 3D structural encoding with hierarchical Q-learning, and employs a surrogate reward model to jointly optimize mutation site selection and amino acid substitution in a multi-step iterative manner. Compared to conventional single-step approaches, the framework significantly expands the explorable sequence space and enables joint sequence–structure modeling. Experimental results demonstrate high accuracy in identifying stabilizing mutations and filtering destabilizing variants on unseen proteins, with strong agreement between predictions and wet-lab validation. This is the first study to combine hierarchical RL with structure-aware GNNs for protein thermal stability design, achieving both computational efficiency and strong generalization across diverse protein targets.

Technology Category

• Machine Learning: Structured Learning
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Designing mutations to optimize protein thermostability remains challenging due to the complex relationship between sequence variations, structural dynamics, and thermostability, often assessed by δδG (the change in free energy of unfolding). Existing methods rely on experimental random mutagenesis or prediction models tested with pre-defined datasets, using sequence-based heuristics and treating enzyme design as a one-step process without iterative refinement, which limits design space exploration and restricts discoveries beyond known variations. We present ThermoRL, a framework based on reinforcement learning (RL) that leverages graph neural networks (GNN) to design mutations with enhanced thermostability. It combines a pre-trained GNN-based encoder with a hierarchical Q-learning network and employs a surrogate model for reward feedback, guiding the RL agent on where (the position) and which (mutant amino acid) to apply for enhanced thermostability. Experimental results show that ThermoRL achieves higher or comparable rewards than baselines while maintaining computational efficiency. It filters out destabilizing mutations and identifies stabilizing mutations aligned with experimental data. Moreover, ThermoRL accurately detects key mutation sites in unseen proteins, highlighting its strong generalizability. This RL-guided approach powered by GNN embeddings offers a robust alternative to traditional protein mutation design.

Problem

• Optimizing protein thermostability via mutation design
• Overcoming limitations of one-step sequence-based heuristic methods
• Enhancing generalizability in mutation site detection for unseen proteins

Innovation

• Reinforcement learning optimizes protein thermostability
• Graph neural networks guide mutation site selection
• Hierarchical Q-learning with surrogate model feedback