A Phylogenetic Approach to Genomic Language Modeling

2025-03-04

AI Summary

Current genomic language models (gLMs) exhibit limited performance in identifying evolutionarily constrained elements in mammalian genomes. To address this, we propose PhyloGPN—a self-supervised genomic language model that explicitly integrates phylogenetic modeling. PhyloGPN is the first gLM to incorporate multi-species whole-genome alignments directly into its loss function, enabling evolutionary signal utilization during training while requiring only a single input sequence—without alignment—at inference time. Furthermore, it constrains nucleotide substitution dynamics using a phylogenetic tree, enhancing biological interpretability. Experiments demonstrate that PhyloGPN significantly outperforms baseline models in predicting functionally disruptive variants and exhibits strong cross-species generalization. By jointly ensuring phylogenetic rigor and practical deployability, PhyloGPN establishes a novel paradigm for interpretable, high-accuracy functional annotation of genomes.

Technology Category

• Machine Learning: Evolutionary Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Search and Optimization: Evolutionary Computation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Genomic language models (gLMs) have shown mostly modest success in identifying evolutionarily constrained elements in mammalian genomes. To address this issue, we introduce a novel framework for training gLMs that explicitly models nucleotide evolution on phylogenetic trees using multispecies whole-genome alignments. Our approach integrates an alignment into the loss function during training but does not require it for making predictions, thereby enhancing the model's applicability. We applied this framework to train PhyloGPN, a model that excels at predicting functionally disruptive variants from a single sequence alone and demonstrates strong transfer learning capabilities.

Problem

• Improving genomic language models for evolutionary constraint identification
• Introducing phylogenetic tree-based training for nucleotide evolution modeling
• Enhancing prediction of functionally disruptive variants using single sequences

Innovation

• Phylogenetic tree modeling for genomic language
• Multispecies alignment integrated in loss function
• Single sequence prediction with transfer learning