Learning Novel Transformer Architecture for Time-series Forecasting

Juyuan Zhang, Wei Zhu, Jiechao Gao
2/19/2025
cs.LGcs.CL

No official code found. Best implementation: josejimenezluna/nnet-ts (108 stars, last commit 2017-10-20).

Abstract

Despite the success of Transformer-based models in the time-series prediction (TSP) tasks, the existing Transformer architecture still face limitations and the literature lacks comprehensive explorations into alternative architectures. To address these challenges, we propose AutoFormer-TS, a novel framework that leverages a comprehensive search space for Transformer architectures tailored to TSP tasks. Our framework introduces a differentiable neural architecture search (DNAS) method, AB-DARTS, which improves upon existing DNAS approaches by enhancing the identification of optimal operations within the architecture. AutoFormer-TS systematically explores alternative attention mechanisms, activation functions, and encoding operations, moving beyond the traditional Transformer design. Extensive experiments demonstrate that AutoFormer-TS consistently outperforms state-of-the-art baselines across various TSP benchmarks, achieving superior forecasting accuracy while maintaining reasonable training efficiency.

View on arXivView PDF

Code Implementations(1)

josejimenezluna/nnet-tsCommunity70%

Neural network architecture for time series forecasting.

10827PythonAug 30, 20159 years agoNOASSERTION
machine-learningneural-networktime-series

Similar Papers

The Power of Architecture: Deep Dive into Transformer Architectures for Long-Term Time Series Forecasting

89% match1 repo152025

Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting

87% match2024

Two Steps Forward and One Behind: Rethinking Time Series Forecasting with Deep Learning

87% match2023

GBT: Two-stage transformer framework for non-stationary time series forecasting

87% match1 repo422023

iTransformer: Inverted Transformers Are Effective for Time Series Forecasting

86% match3 repos2.2k2026

iTransformer: Inverted Transformers Are Effective for Time Series Forecasting

86% match4 repos2.2k2023

Cite this paper

@article{zhang2025learning,
  title  = {Learning Novel Transformer Architecture for Time-series Forecasting},
  author = {Juyuan Zhang and Wei Zhu and Jiechao Gao},
  year   = {2025},
  eprint = {2502.13721},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.13721v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.13721.svg)](https://codeofpaper.com/paper/2502.13721)

Discussion