Advancing bioinformatics with large language models: components, applications and perspectives

2024-01-08

AI Summary

The application of large language models (LLMs) in bioinformatics remains fragmented, lacking a unified methodological framework across omics domains. Method: We propose the first comprehensive LLM framework spanning multi-omics—including genomics, transcriptomics, proteomics, single-cell biology, and drug discovery—featuring a biologically informed tokenization scheme and pretraining paradigm tailored to biomolecular sequences. Our approach integrates state-of-the-art biomedical foundation models (e.g., ESM, DNABERT, scGPT) within a Transformer architecture, enhanced via self-supervised learning to strengthen sequence representation learning. Contributions: (1) A systematic, taxonomy-based mapping of 100+ LLM-driven bioinformatics applications; (2) identification and analysis of critical bottlenecks—including cross-omics transferability and data sparsity; and (3) a practitioner-oriented, developer–user co-optimization guideline. This work advances reproducible, scalable AI-for-Science deployment in precision medicine and systems biology.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models (LLMs) are a class of artificial intelligence models based on deep learning, which have great performance in various tasks, especially in natural language processing (NLP). Large language models typically consist of artificial neural networks with numerous parameters, trained on large amounts of unlabeled input using self-supervised or semi-supervised learning. However, their potential for solving bioinformatics problems may even exceed their proficiency in modeling human language. In this review, we will provide a comprehensive overview of the essential components of large language models (LLMs) in bioinformatics, spanning genomics, transcriptomics, proteomics, drug discovery, and single-cell analysis. Key aspects covered include tokenization methods for diverse data types, the architecture of transformer models, the core attention mechanism, and the pre-training processes underlying these models. Additionally, we will introduce currently available foundation models and highlight their downstream applications across various bioinformatics domains. Finally, drawing from our experience, we will offer practical guidance for both LLM users and developers, emphasizing strategies to optimize their use and foster further innovation in the field.

Problem

• Large Language Models
• Bioinformatics
• Scientific Research Advancement

Innovation

• Large Language Models
• Bioinformatics
• Drug Discovery