GoBERT: Gene Ontology Graph Informed BERT for Universal Gene Function Prediction

2025-01-03

AI Summary

Existing gene function prediction methods predominantly rely on protein structural or family information, limiting their capacity for generalizable inference using gene knowledge graphs. Method: We propose GO-BERT—the first framework integrating Gene Ontology (GO) graph structure with BERT-style language modeling. It introduces two novel, GO-graph-guided pretraining tasks—neighborhood relation prediction and functional specificity masked token recovery—to jointly model explicit and implicit functional associations, enabling zero-shot discovery of novel functions. The architecture employs graph-aware embeddings and multi-label self-supervised learning to jointly encode functional semantics of genes and their products. Contribution/Results: GO-BERT achieves significant improvements over state-of-the-art methods across multi-species benchmarks. Biological case studies confirm its ability to accurately predict previously unannotated functions, and ablation studies validate the critical roles of GO graph structure and the dual pretraining task design.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Exploring the functions of genes and gene products is crucial to a wide range of fields, including medical research, evolutionary biology, and environmental science. However, discovering new functions largely relies on expensive and exhaustive wet lab experiments. Existing methods of automatic function annotation or prediction mainly focus on protein function prediction with sequence, 3D-structures or protein family information. In this study, we propose to tackle the gene function prediction problem by exploring Gene Ontology graph and annotation with BERT (GoBERT) to decipher the underlying relationships among gene functions. Our proposed novel function prediction task utilizes existing functions as inputs and generalizes the function prediction to gene and gene products. Specifically, two pre-train tasks are designed to jointly train GoBERT to capture both explicit and implicit relations of functions. Neighborhood prediction is a self-supervised multi-label classification task that captures the explicit function relations. Specified masking and recovering task helps GoBERT in finding implicit patterns among functions. The pre-trained GoBERT possess the ability to predict novel functions for various gene and gene products based on known functional annotations. Extensive experiments, biological case studies, and ablation studies are conducted to demonstrate the superiority of our proposed GoBERT.

Problem

• Gene Function Prediction
• Knowledge Graph
• Protein Structure

Innovation

• GoBERT
• Gene Ontology Knowledge Graph
• Predictive Performance