Meta-Learning for Cross-Task Generalization in Protein Mutation Property Prediction

2025-10-23

AI Summary

Weak cross-dataset generalization in protein mutation property prediction—stemming from experimental condition heterogeneity and scarcity of target-domain data—remains a critical challenge. To address this, we introduce model-agnostic meta-learning (MAML) to this task for the first time and propose a separator-token-based mutation encoding strategy that explicitly injects mutation site information into the Transformer’s sequence context, thereby overcoming the standard Transformer’s limitation in modeling local mutations. Our method enables rapid few-step adaptation across three heterogeneous datasets—functional adaptability, thermal stability, and solubility. It achieves 29–94% higher cross-task prediction accuracy and 55–65% faster training compared to conventional fine-tuning. Crucially, its generalization performance is independent of source-domain dataset size, significantly enhancing the robustness and practical applicability of mutation prediction models.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Metareasoning and Metaheuristics
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Protein mutations can have profound effects on biological function, making accurate prediction of property changes critical for drug discovery, protein engineering, and precision medicine. Current approaches rely on fine-tuning protein-specific transformers for individual datasets, but struggle with cross-dataset generalization due to heterogeneous experimental conditions and limited target domain data. We introduce two key innovations: (1) the first application of Model-Agnostic Meta-Learning (MAML) to protein mutation property prediction, and (2) a novel mutation encoding strategy using separator tokens to directly incorporate mutations into sequence context. We build upon transformer architectures integrating them with MAML to enable rapid adaptation to new tasks through minimal gradient steps rather than learning dataset-specific patterns. Our mutation encoding addresses the critical limitation where standard transformers treat mutation positions as unknown tokens, significantly degrading performance. Evaluation across three diverse protein mutation datasets (functional fitness, thermal stability, and solubility) demonstrates significant advantages over traditional fine-tuning. In cross-task evaluation, our meta-learning approach achieves 29% better accuracy for functional fitness with 65% less training time, and 94% better accuracy for solubility with 55% faster training. The framework maintains consistent training efficiency regardless of dataset size, making it particularly valuable for industrial applications and early-stage protein design where experimental data is limited. This work establishes a systematic application of meta-learning to protein mutation analysis and introduces an effective mutation encoding strategy, offering transformative methodology for cross-domain generalization in protein engineering.

Problem

• Predicting protein mutation property changes across diverse experimental conditions
• Overcoming limited target domain data in protein mutation analysis
• Improving cross-dataset generalization for protein engineering applications

Innovation

• Model-Agnostic Meta-Learning for protein mutation prediction
• Mutation encoding with separator tokens in sequences
• Transformer integration for rapid cross-task adaptation