Can Tabular In-Context Learners Generalize to Biomolecular Property Prediction?

2026-06-30

AI Summary

This work proposes leveraging tabular foundation models—such as TabPFN3 and TabICL—as in-context learners for few-shot biomolecular property prediction under limited labeled data. By integrating pretrained biological sequence representations, specifically ESMC protein embeddings and ECFP/RDKit molecular descriptors, the approach demonstrates that tabular context learners pretrained on synthetic causal graphs can effectively generalize to both protein and small-molecule tasks. The study highlights the critical role of representation choice in predictive performance. Evaluated on standard benchmarks including ProteinGym and TDC ADMET, the method achieves state-of-the-art results in protein fitness regression and matches specialized models in small-molecule classification, underscoring its versatility and competitive efficacy across diverse biomolecular domains.

Abstract

Predicting biomolecular properties from limited labeled data is a central bottleneck in protein engineering and small-molecule design. As strong pretrained encoders now supply rich fixed-length representations, the difficulty has shifted from representation learning to building a data-efficient predictor for the few-shot regime. Tabular foundation models such as TabPFN3 and TabICL are unlikely candidates for this role: they are in-context learners pretrained on synthetic tables drawn from random causal graphs, a generative prior with no obvious correspondence to the processes that produce protein sequences or molecular graphs. That this tabular, causal inductive bias should transfer to biomolecular data at all is unintuitive, yet we find it does. Treating each method as a predictor-representation pair, we evaluate across two domains. Over a fixed ESMC representation, tabular in-context learning is consistently competitive for protein fitness regression on ProteinGym and a diverse esterase dataset. For small-molecule classification with ECFP/RDKit descriptors, no single pairing dominates across TDC ADMET, MoleculeNet, FS-Mol, and DrugOOD; representation choice becomes a primary determinant, as expected when the predictor's own prior is indifferent to molecular structure. We conclude that tabular foundation models are strong performers on biomolecular prediction tasks, but that their performance depends strongly on the sequence or molecular representation used.

Problem

• biomolecular property prediction
• few-shot learning
• protein engineering
• small-molecule design
• data-efficient prediction

Innovation

• tabular in-context learning
• biomolecular property prediction
• foundation models
• few-shot learning
• representation transfer