Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset

2025-11-29

AI Summary

Existing protein–ligand affinity prediction models rely on simplified datasets that fail to reflect biologically realistic protein modifications—such as substitutions, insertions, deletions, and phosphorylation—common in drug discovery. Method: We introduce the first kinase-modification–focused extension of the DAVIS dataset (4,032 kinase–ligand pairs) and propose three novel benchmark tasks: enhanced-data prediction, wild-type-to-modified generalization, and few-shot modified-kinase generalization, systematically evaluating model robustness and generalization under modification perturbations. Leveraging molecular interaction-aware data restructuring, we jointly assess both docking-based and non-docking (e.g., ML-based) methods. Results: Docking-based models demonstrate superior zero-shot generalization to unseen modifications, whereas non-docking models achieve substantial performance gains after fine-tuning on only a few modified-kinase examples. This work establishes a reproducible, biologically grounded evaluation framework for affinity prediction under realistic post-translational and mutational perturbations, advancing precision drug discovery.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Other Foundations of Data Mining & Knowledge Management
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Advancements in AI for science unlocks capabilities for critical drug discovery tasks such as protein-ligand binding affinity prediction. However, current models overfit to existing oversimplified datasets that does not represent naturally occurring and biologically relevant proteins with modifications. In this work, we curate a complete and modification-aware version of the widely used DAVIS dataset by incorporating 4,032 kinase-ligand pairs involving substitutions, insertions, deletions, and phosphorylation events. This enriched dataset enables benchmarking of predictive models under biologically realistic conditions. Based on this new dataset, we propose three benchmark settings-Augmented Dataset Prediction, Wild-Type to Modification Generalization, and Few-Shot Modification Generalization-designed to assess model robustness in the presence of protein modifications. Through extensive evaluation of both docking-free and docking-based methods, we find that docking-based model generalize better in zero-shot settings. In contrast, docking-free models tend to overfit to wild-type proteins and struggle with unseen modifications but show notable improvement when fine-tuned on a small set of modified examples. We anticipate that the curated dataset and benchmarks offer a valuable foundation for developing models that better generalize to protein modifications, ultimately advancing precision medicine in drug discovery. The benchmark is available at: https://github.com/ZhiGroup/DAVIS-complete

Problem

• Addresses overfitting in protein-ligand affinity prediction models
• Introduces a modification-aware dataset for biologically realistic benchmarking
• Evaluates model robustness against protein modifications in drug discovery

Innovation

• Curated a complete modification-aware DAVIS dataset with kinase-ligand pairs
• Proposed three benchmark settings to assess model robustness to modifications
• Evaluated docking-free and docking-based methods for generalization on modifications