SAITS: Self-Attention-based Imputation for Time Series

Wenjie Du, David Cote, Yan Liu
2/17/2022
cs.LG

Official code: WenjieDu/SAITS (513 stars, MIT, PyTorch, last commit 2025-10-01). 1 more implementation below.

Abstract

Missing data in time series is a pervasive problem that puts obstacles in the way of advanced analysis. A popular solution is imputation, where the fundamental challenge is to determine what values should be filled in. This paper proposes SAITS, a novel method based on the self-attention mechanism for missing value imputation in multivariate time series. Trained by a joint-optimization approach, SAITS learns missing values from a weighted combination of two diagonally-masked self-attention (DMSA) blocks. DMSA explicitly captures both the temporal dependencies and feature correlations between time steps, which improves imputation accuracy and training speed. Meanwhile, the weighted-combination design enables SAITS to dynamically assign weights to the learned representations from two DMSA blocks according to the attention map and the missingness information. Extensive experiments quantitatively and qualitatively demonstrate that SAITS outperforms the state-of-the-art methods on the time-series imputation task efficiently and reveal SAITS' potential to improve the learning performance of pattern recognition models on incomplete time-series data from the real world. The code is open source on GitHub at https://github.com/WenjieDu/SAITS.

View on arXivView PDF

Code Implementations(2)

WenjieDu/SAITSOfficial100%
PyTorchMIT

The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516

51370Shell, PythonDec 7, 20211 years agoMIT
attentionattention-mechanismdeep-learningimputationimputation-model+15 more
WenjieDu/PyPOTSCommunity100%
PyTorchBSD-3-Clause

A Python toolkit/library for reality-centric machine/deep learning & data mining on partially-observed time series, with 50+ SOTA neural network models for scientific analysis tasks (imputation, classification, clustering, forecasting, anomaly detection, cleaning) on incomplete industrial irregularly-sampled multivariate TS with NaN missing values

2,060190Shell, PythonMar 29, 20225 months agoBSD-3-Clause
anomaly-detectionclassificationclusteringdata-analysisdata-mining+10 more

Cite this paper

@article{du2022saits,
  title  = {SAITS: Self-Attention-based Imputation for Time Series},
  author = {Wenjie Du and David Cote and Yan Liu},
  year   = {2022},
  eprint = {2202.08516},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2202.08516}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2202.08516.svg)](https://codeofpaper.com/paper/2202.08516)

Discussion