Transformers Meet Relational Databases

Jakub Peleška, Gustav Šír
12/6/2024
cs.LGcs.DB

No official code found. Best implementation: jakubpeleska/deep-db-learning (7 stars, last commit 2024-12-09).

Abstract

Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their extension to the more general case of relational databases. In this paper, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular Transformers from database storage systems. We address the challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results demonstrate a superior performance of this newly proposed class of neural architectures.

View on arXivView PDF

Code Implementations(1)

jakubpeleska/deep-db-learningCommunity100%

A study on integrating transformer architectures with relational databases via a modular message-passing framework, demonstrating enhanced performance.

71Python, Jupyter NotebookJan 15, 20241 years ago

Similar Papers

The Duck's Brain: Training and Inference of Neural Networks in Modern Database Engines

84% match2023

SPARE: A Single-Pass Neural Model for Relational Databases

82% match2023

Neuro-Relational Programs: Unifying Queries and Neural Computation over Structured Data

82% match2026

Database Views as Explanations for Relational Deep Learning

80% match1 repo2025

Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data

79% match1 repo912025

Transformers are Graph Neural Networks

79% match1 repo552025

Cite this paper

@article{peleka2024transformers,
  title  = {Transformers Meet Relational Databases},
  author = {Jakub Peleška and Gustav Šír},
  year   = {2024},
  eprint = {2412.05218},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.05218v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.05218.svg)](https://codeofpaper.com/paper/2412.05218)

Discussion