🤖 AI Summary
Current computational assessments of antimicrobial peptides (AMPs) are largely confined to binary classification tasks and lack a unified, homology-controlled benchmark that jointly evaluates critical experimental endpoints such as potency, antimicrobial spectrum, hemolytic activity, toxicity, and selectivity. To address this gap, this work proposes AMPBench-MT, a comprehensive multi-task benchmark that integrates AMP identification, species-specific pMIC regression, and multiple safety-related endpoints under strict sequence homology control. Leveraging protein language model embeddings, graph neural networks, and a multi-task learning framework, the study systematically evaluates 161 endpoints and finds that frozen language model embeddings achieve superior performance in pMIC prediction. Importantly, the results demonstrate that high identification accuracy does not reliably translate to strong experimental performance, advocating for a paradigm shift in AMP evaluation from recognition-oriented approaches toward endpoint-aware, evidence-based auditing.
📝 Abstract
Computational AMP discovery is often evaluated through AMP/non-AMP recognition, yet follow-up decisions depend on assay-derived evidence such as target-species potency, hemolysis, toxicity, and selectivity. Existing AMP and peptide benchmarks cover binary recognition, multilabel annotation, assay regression, or broader peptide-model comparison, but they do not jointly place AMP recognition, species-conditioned potency, spectrum, safety-facing proxy endpoints, and cross-endpoint behavior within one sequence-homology-controlled protocol. To address this problem, we introduce AMPBench-MT, a provenance-preserving benchmark that standardizes canonical peptide records and organizes them into binary recognition, species-conditioned pMIC regression, and endpoint-specific potency and safety-facing readouts. Across 161 endpoint-specific model evaluations, high binary performance does not reliably indicate assay-endpoint behavior. Frozen protein-language-model embeddings form the leading pMIC error cluster, while graph and classical regressors remain close. Spectrum labels further reveal that PR-oriented metrics can be misleading under scarce observed negatives, whereas low-toxicity, HC50 hemolysis, and selectivity expose smaller but more assay-facing signals. AMPBench-MT shows that AMP evaluation should move beyond recognition leaderboards toward endpoint-aware evidence auditing. Our proposed benchmark is available at https://huggingface.co/datasets/ZihengZhou06/AMPBench-MT.