Molecule Attention Transformer

Łukasz Maziarka, Tomasz Danel, Sławomir Mucha, Krzysztof Rataj, Jacek Tabor +1 more
2/19/2020
cs.LGphysics.comp-phstat.ML

Official code: ardigen/MAT (254 stars, MIT, last commit 2020-04-01).

Abstract

Designing a single neural network architecture that performs competitively across a range of molecule property prediction tasks remains largely an open challenge, and its solution may unlock a widespread use of deep learning in the drug discovery industry. To move towards this goal, we propose Molecule Attention Transformer (MAT). Our key innovation is to augment the attention mechanism in Transformer using inter-atomic distances and the molecular graph structure. Experiments show that MAT performs competitively on a diverse set of molecular prediction tasks. Most importantly, with a simple self-supervised pretraining, MAT requires tuning of only a few hyperparameter values to achieve state-of-the-art performance on downstream tasks. Finally, we show that attention weights learned by MAT are interpretable from the chemical point of view.

View on arXivView PDF

Code Implementations(1)

ardigen/MATOfficial100%
MIT

The official implementation of the Molecule Attention Transformer.

25462Python, Jupyter NotebookOct 14, 20196 years agoMIT

Similar Papers

Neural Message Passing for Quantum Chemistry

88% match1 repo2392017

Synergistic Fusion of Graph and Transformer Features for Enhanced Molecular Property Prediction

88% match2023

FunQG: Molecular Representation Learning Via Quotient Graphs

86% match1 repo122022

Analyzing Learned Molecular Representations for Property Prediction

86% match1 repo2.5k2019

Path-Augmented Graph Transformer Network

86% match1 repo442019

Self-Supervised Graph Transformer on Large-Scale Molecular Data

86% match1 repo3942020

Cite this paper

@article{maziarka2020molecule,
  title  = {Molecule Attention Transformer},
  author = {Łukasz Maziarka and Tomasz Danel and Sławomir Mucha and Krzysztof Rataj and Jacek Tabor and Stanisław Jastrzębski},
  year   = {2020},
  eprint = {2002.08264},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2002.08264}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2002.08264.svg)](https://codeofpaper.com/paper/2002.08264)

Discussion