structural model evaluation

Developing cross-platform graphical user interfaces (GUIs) and plugins for command-line tools in structural bioinformatics is often costly and complex. This work proposes a three-stage automated workflow that leverages a platform-agnostic formal GUI specification, decouples model, view, and presenter components through the Model–View–Presenter (MVP) architectural pattern, and employs a dedicated code generator to automatically produce plugins for target platforms—namely VMD, PyMOL, and the web. To the best of our knowledge, this is the first systematic application of the MVP pattern to the automatic GUI generation for CLI tools, substantially enhancing logic reusability, cross-platform portability, and development efficiency. The framework’s generality, extensibility, and practical utility are demonstrated by successfully generating plugins for multiple tools from the Structural Bioinformatics Library across all three platforms.

Recent advances in de novo protein binder design have enabled increasing experimental validation, yet reported in silico metrics remain difficult to interpret or compare across studies due to non-standardized evaluation protocols. We introduce ProtDBench, a standardized and throughput-aware evaluation framework for protein binder design. ProtDBench defines unified benchmark tasks, evaluation protocols, and success criteria, enabling systematic analysis of how evaluation design influences observed performance. Using a large wet-lab annotated dataset, we analyze commonly used structure prediction models as evaluation verifiers, revealing substantial verifier-dependent bias and limited agreement under identical filtering protocols. We then benchmark representative open-source generative binder design methods across ten diverse protein targets under a fixed evaluation protocol. Beyond per-sequence success rates, ProtDBench incorporates throughput-aware metrics based on a fixed 24-hour budget, as well as cluster-level success criteria to account for structural diversity. Together, these results expose systematic differences induced by filtering rules, success definitions, and throughput-aware evaluation between computational efficiency, success rate, and structural diversity. Overall, ProtDBench provides a fair and reproducible evaluation pipeline that supports systematic and controlled comparison of protein binder design methods under realistic evaluation settings.

Long-standing limitations in de novo protein design—including the absence of standardized, publicly available benchmark datasets and multidimensional evaluation frameworks—have hindered fair, reproducible comparison across methods. To address this, we introduce PDFBench, the first function-driven comprehensive benchmark supporting both description-guided and keyword-guided design tasks. It features an integrated evaluation framework comprising 22 metrics across five dimensions: sequence validity, structural fidelity, language–protein alignment (a novel dimension introduced herein), novelty, and diversity. We systematically analyze inter-dimensional correlations and provide principled metric selection guidelines. Leveraging sequence modeling, AlphaFold2-based structural validation, natural language understanding, and diversity quantification, we rigorously evaluate five state-of-the-art models, revealing their fundamental capabilities and limitations. All data, code, and evaluation pipelines are fully open-sourced to foster community advancement.

Computational structural predictions often introduce bias in fitness prediction for variants in intrinsically disordered regions (IDRs), undermining reliability. Method: We propose a zero-shot, fine-tuning-free multimodal framework that jointly leverages pretrained protein language models (e.g., ESM) and AlphaFold2-predicted structures—while critically emphasizing the necessity of experimentally determined structures for accurate modeling. A simple yet effective multimodal integration enables sequence–structure co-modeling. Contributions/Results: (1) We systematically demonstrate that computational structure prediction exacerbates bias in IDRs and establish that high-resolution experimental structures are decisive for robust fitness prediction; (2) we identify structural quality and local disorder-to-order propensity as key determinants of zero-shot performance; (3) on the ProteinGym benchmark, our lightweight multimodal ensemble establishes a strong, plug-and-play zero-shot baseline—enabling reliable interpretation of genetic variants and supporting rational protein engineering without task-specific adaptation.

Current antibody affinity evaluation methods typically analyze antibody sequences or structures in isolation, lacking a unified benchmark that treats the antibody–antigen (Ab–Ag) complex as the functional unit and reflects true binding capability. To address this, we propose AbBiBench—the first function-oriented evaluation framework grounded in complex likelihood estimation, breaking from conventional single-antibody assessment paradigms. AbBiBench integrates masked language modeling, autoregressive generation, inverse folding, diffusion-based structure generation, and geometric graph neural networks, jointly scoring candidates across experimental affinity, structural integrity, and biophysical properties. We systematically evaluate 14 state-of-the-art models on a benchmark comprising 9 antigens and 156,000 antibody variants. Results show that structure-conditioned inverse folding models achieve top performance. In an H1N1 antibody design case study, AbBiBench demonstrates strong predictive validity: model-derived complex likelihood correlates significantly with experimental dissociation constants (K_D; Pearson *r* = 0.72).

Current single-pass decoding approaches for protein sequence design struggle to simultaneously achieve high fidelity in both structure and sequence, limiting design success rates. This work proposes a novel multi-objective search framework that, for the first time, integrates large language models (e.g., o4-mini, Gemini-3) as generative optimizers and incorporates vision-language models to provide multimodal feedback based on structural images. By synergistically combining RosettaFold3 for structure prediction with ProteinMPNN and LigandMPNN for sequence generation, the framework enables a controllable balance between exploration and exploitation. Evaluated on 400 suboptimal sequences, the method improves structural fidelity by 18%–68% and increases design success rates by 2.5-fold, demonstrating strong generalization across independent test sets and de novo backbones.

Current biomolecular sequence design methods lack unified, reproducible evaluation standards, hindering fair and rigorous performance comparison. To address this, we introduce BioSeqEval—a modular, open-source Python evaluation library that systematically integrates three model-agnostic metric categories: sequence-based, embedding-based, and property-based—representing the first such comprehensive framework. It supports one-shot and iterative design evaluation across diverse sequence modalities, including small molecules, DNA, RNA, peptides, and proteins. The library incorporates state-of-the-art pretrained embedding models, machine learning–based property predictors, efficient sequence alignment tools, and interactive visualization modules for diagnostic analysis. Empirical evaluation demonstrates that BioSeqEval significantly enhances evaluation standardization, cross-method comparability, and methodological transparency. It exhibits strong flexibility and robustness across multiple benchmark design tasks, enabling reproducible, interpretable, and scalable assessment of generative sequence models.

This study addresses the scarcity of structured, context-rich experimental data in targeted protein degradation (TPD), which has hindered the development of computational models. To overcome this limitation, the authors propose the first expert-in-the-loop large language model (LLM) agent framework tailored for TPD. By integrating lightweight prompt optimization, terminology-aware transfer, and a triangulation-based validation mechanism, the framework automatically extracts multidimensional information—including compounds, targets, recruiters, and critical experimental conditions—from scientific literature. Requiring only minimal annotated data, it achieves high-accuracy cross-task transfer. The resulting molecular glue and PROTAC databases are expanded by 81% and 92%, respectively, with expert-validated accuracy rates of 92% and 82.5%, substantially enhancing condition-aware modeling of degrader activity.

Existing protein language models struggle to accurately map natural language questions to executable 3D structural measurements. This work introduces ProtStructQA—the first protein structure question-answering benchmark grounded in an executable domain-specific language (DSL)—where questions and answers are generated by running latent DSL programs on AlphaFold-predicted structures. The study reframes scientific question answering as a “language-to-structure-measurement” compilation process, integrating grammar-constrained voting, chain-of-thought reasoning, and ReAct-style tool invocation. Evaluation on 382.2K questions reveals a referential competence threshold in Qwen3 and Gemma-3 models within the 1.7B–4B parameter range, which significantly influences the effectiveness of their reasoning strategies.

The internal mechanisms of protein structure prediction models—such as AlphaFold2 and OpenFold—remain largely opaque, hindering model diagnosis, optimization, and biological interpretation. Method: This work introduces an explainability framework based on ablation analysis and component-wise contribution quantification to systematically assess the impact of individual OpenFold modules on prediction accuracy. Using multi-scale metrics—including pLDDT and RMSD—we evaluate module contributions across a diverse set of protein targets and analyze their dependence on sequence length. Contribution/Results: We reveal pronounced protein-specificity in the contributions of core modules (e.g., Evoformer, Structure Module), with several components exhibiting nonlinear sensitivity to protein length. This study presents the first fine-grained, component-level attribution analysis of OpenFold, delivering a reproducible methodological framework and empirical evidence to support model debugging, lightweight architecture design, and enhanced biological interpretability.