Pretraining is All You Need: A Multi-Atlas Enhanced Transformer Framework for Autism Spectrum Disorder Classification

Lucas Mahler, Qi Wang, Julius Steiglechner, Florian Birk, Samuel Heczko +2 more
7/4/2023
cs.CVcs.LGeess.IV

Official code: Lugges991/METAFormer (10 stars, last commit 2023-08-31).

Abstract

Autism spectrum disorder (ASD) is a prevalent psychiatric condition characterized by atypical cognitive, emotional, and social patterns. Timely and accurate diagnosis is crucial for effective interventions and improved outcomes in individuals with ASD. In this study, we propose a novel Multi-Atlas Enhanced Transformer framework, METAFormer, ASD classification. Our framework utilizes resting-state functional magnetic resonance imaging data from the ABIDE I dataset, comprising 406 ASD and 476 typical control (TC) subjects. METAFormer employs a multi-atlas approach, where flattened connectivity matrices from the AAL, CC200, and DOS160 atlases serve as input to the transformer encoder. Notably, we demonstrate that self-supervised pretraining, involving the reconstruction of masked values from the input, significantly enhances classification performance without the need for additional or separate training data. Through stratified cross-validation, we evaluate the proposed framework and show that it surpasses state-of-the-art performance on the ABIDE I dataset, with an average accuracy of 83.7% and an AUC-score of 0.832. The code for our framework is available at https://github.com/Lugges991/METAFormer

View on arXivView PDF

Code Implementations(1)

Lugges991/METAFormerOfficial100%

Repository to the paper "Pretraining is all you need: A Multi-Atlas Transformer Framework for Autism Spectrum Disorder Classification"

103PythonJun 30, 20233 years ago

Similar Papers

ASDFormer: A Transformer with Mixtures of Pooling-Classifier Experts for Robust Autism Diagnosis and Biomarker Discovery

86% match2025

Improving Brain Disorder Diagnosis with Advanced Brain Function Representation and Kolmogorov-Arnold Networks

85% match1 repo32025

Community-Aware Transformer for Autism Prediction in fMRI Connectome

84% match1 repo472023

MMASD+: A Novel Dataset for Privacy-Preserving Behavior Analysis of Children with Autism Spectrum Disorder

83% match1 repo192024

An Evaluation of Machine Learning Approaches for Early Diagnosis of Autism Spectrum Disorder

82% match1 repo32023

Self-supervised Graph Transformer with Contrastive Learning for Brain Connectivity Analysis towards Improving Autism Detection

81% match2025

Cite this paper

@article{mahler2023pretraining,
  title  = {Pretraining is All You Need: A Multi-Atlas Enhanced Transformer Framework for Autism Spectrum Disorder Classification},
  author = {Lucas Mahler and Qi Wang and Julius Steiglechner and Florian Birk and Samuel Heczko and Klaus Scheffler and Gabriele Lohmann},
  year   = {2023},
  eprint = {2307.01759},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2307.01759v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.01759.svg)](https://codeofpaper.com/paper/2307.01759)

Discussion