Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language

Philipp Seidl, Andreu Vall, Sepp Hochreiter, Günter Klambauer
3/6/2023
q-bio.BMcs.CLcs.LG

No official code found. Best implementation: ml-jku/clamp (116 stars, last commit 2026-02-26).

Abstract

Activity and property prediction models are the central workhorses in drug discovery and materials sciences, but currently they have to be trained or fine-tuned for new tasks. Without training or fine-tuning, scientific language models could be used for such low-data tasks through their announced zero- and few-shot capabilities. However, their predictive quality at activity prediction is lacking. In this work, we envision a novel type of activity prediction model that is able to adapt to new prediction tasks at inference time, via understanding textual information describing the task. To this end, we propose a new architecture with separate modules for chemical and natural language inputs, and a contrastive pre-training objective on data from large biochemical databases. In extensive experiments, we show that our method CLAMP yields improved predictive performance on few-shot learning benchmarks and zero-shot problems in drug discovery. We attribute the advances of our method to the modularized architecture and to our pre-training objective.

View on arXivView PDF

Code Implementations(1)

ml-jku/clampCommunity100%

Code for the paper Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language

11610Python, Jupyter NotebookJan 22, 20237 months agoNOASSERTION
assay-modelingcheminformaticscontrastive-learningdrug-discoverymachine-learning+2 more

Similar Papers

Target-Free Compound Activity Prediction via Few-Shot Learning

86% match2023

A Hitchhiker's Guide to Deep Chemical Language Processing for Bioactivity Prediction

86% match1 repo112024

MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction

84% match1 repo392024

Reasoning-Enhanced Large Language Models for Molecular Property Prediction

83% match1 repo32025

Conditional Chemical Language Models are Versatile Tools in Drug Discovery

83% match2025

Enhancing Drug-Target Interaction Prediction through Transfer Learning from Activity Cliff Prediction Tasks

83% match1 repo22024

Cite this paper

@article{seidl2023enhancing,
  title  = {Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language},
  author = {Philipp Seidl and Andreu Vall and Sepp Hochreiter and Günter Klambauer},
  year   = {2023},
  eprint = {2303.03363},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.03363v2},
  journal = {ICML 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.03363.svg)](https://codeofpaper.com/paper/2303.03363)

Discussion