ProtChatGPT: Towards Understanding Proteins with Large Language Models

2024-02-15

arXiv.org

AI Summary

This work addresses the challenge of interpretable modeling of structure–function relationships in proteins. To this end, we propose ProteinChat—the first natural language–interfaced protein structure understanding system—built upon a cross-modal architecture that integrates a multi-scale protein encoder with a novel Protein-Language Pertaining Transformer (PLP-former). We further introduce learnable projection adapters to align protein representations semantically with large language models (e.g., LLaMA). The entire system is trained end-to-end via joint optimization. On multiple protein-centric question-answering and functional reasoning benchmarks, ProteinChat significantly outperforms existing baselines, generating answers that are both highly accurate and intrinsically interpretable. To foster reproducibility and community advancement, we publicly release the source code, pre-trained models, and benchmark datasets.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Protein research is crucial in various fundamental disciplines, but understanding their intricate structure-function relationships remains challenging. Recent Large Language Models (LLMs) have made significant strides in comprehending task-specific knowledge, suggesting the potential for ChatGPT-like systems specialized in protein to facilitate basic research. In this work, we introduce ProtChatGPT, which aims at learning and understanding protein structures via natural languages. ProtChatGPT enables users to upload proteins, ask questions, and engage in interactive conversations to produce comprehensive answers. The system comprises protein encoders, a Protein-Language Pertaining Transformer (PLP-former), a projection adapter, and an LLM. The protein first undergoes protein encoders and PLP-former to produce protein embeddings, which are then projected by the adapter to conform with the LLM. The LLM finally combines user questions with projected embeddings to generate informative answers. Experiments show that ProtChatGPT can produce promising responses to proteins and their corresponding questions. We hope that ProtChatGPT could form the basis for further exploration and application in protein research. Code and our pre-trained model will be publicly available.

Problem

• Protein Structure
• Function Relationship
• Biological Science

Innovation

• ProtChatGPT
• PLP-former
• Natural Language Processing