Design Proteins Using Large Language Models: Enhancements and Comparative Analyses

Kamyar Zeinalipour, Neda Jamshidi, Monica Bianchini, Marco Maggini, Marco Gori
8/12/2024
q-bio.QMcs.AIcs.LG

No official code found. Best implementation: KamyarZeinalipour/protein-design-LLMs (7 stars, MIT, last commit 2024-12-12).

Abstract

Pre-trained LLMs have demonstrated substantial capabilities across a range of conventional natural language processing (NLP) tasks, such as summarization and entity recognition. In this paper, we explore the application of LLMs in the generation of high-quality protein sequences. Specifically, we adopt a suite of pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and gemma-7B4, to produce valid protein sequences. All of these models are publicly available.5 Unlike previous work in this field, our approach utilizes a relatively small dataset comprising 42,000 distinct human protein sequences. We retrain these models to process protein-related data, ensuring the generation of biologically feasible protein structures. Our findings demonstrate that even with limited data, the adapted models exhibit efficiency comparable to established protein-focused models such as ProGen varieties, ProtGPT2, and ProLLaMA, which were trained on millions of protein sequences. To validate and quantify the performance of our models, we conduct comparative analyses employing standard metrics such as pLDDT, RMSD, TM-score, and REU. Furthermore, we commit to making the trained versions of all four models publicly available, fostering greater transparency and collaboration in the field of computational biology.

View on arXivView PDF

Code Implementations(1)

KamyarZeinalipour/protein-design-LLMsCommunity100%
MIT

This repository contains a Python script designed to generate protein sequences using fine-tuned large language models (LLMs). The script leverages models such as Mistral-7B, Llama-2-7B, Llama-3-8B, and Gemma-7B, which have been adapted for protein design tasks.

72PythonJul 11, 20241 years agoMIT

Similar Papers

Protein Large Language Models: A Comprehensive Survey

92% match1 repo1932025

Computational Protein Science in the Era of Large Language Models (LLMs)

92% match2025

A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understanding

92% match1 repo102024

Structure-informed Language Models Are Protein Designers

91% match1 repo2202023

ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding

90% match1 repo342024

ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training

89% match1 repo512024

Cite this paper

@article{zeinalipour2024design,
  title  = {Design Proteins Using Large Language Models: Enhancements and Comparative Analyses},
  author = {Kamyar Zeinalipour and Neda Jamshidi and Monica Bianchini and Marco Maggini and Marco Gori},
  year   = {2024},
  eprint = {2408.06396},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.06396v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.06396.svg)](https://codeofpaper.com/paper/2408.06396)

Discussion