Fine-Tuning a Time Series Foundation Model with Wasserstein Loss

Andrei Chernov
9/18/2024
cs.LGcs.AIcs.CL

No official code found. Best implementation: ChernovAndrey/chronos-forecasting-wasserstein (4 stars, Apache-2.0, last commit 2024-11-20).

Abstract

Inspired by recent advancements in large language models (LLMs) for Natural Language Processing (NLP), there has been a surge in research focused on developing foundational models for time series forecasting. One approach involves training LLM architectures on tokenized time series data using cross-entropy loss. Although this method has demonstrated promising results, cross-entropy loss is primarily designed for classification tasks and does not account for the distance between classes. To address this limitation, we propose using the Wasserstein loss for such architectures. To validate our approach, we fine-tuned a foundational time series model on $22$ zero-shot datasets, comparing the performance of cross-entropy loss with that of Wasserstein loss. Our results demonstrate that replacing cross-entropy loss with Wasserstein loss significantly improves point estimation.

View on arXivView PDF

Code Implementations(1)

ChernovAndrey/chronos-forecasting-wassersteinCommunity100%
Apache-2.0

Chronos: Pretrained (Language) Models for Probabilistic Time Series Forecasting.

41PythonOct 27, 20241 years agoApache-2.0

Similar Papers

Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities

84% match1 repo172024

Adapting Large Language Models for Time Series Modeling via a Novel Parameter-efficient Adaptation Method

83% match2025

Are Language Models Actually Useful for Time Series Forecasting?

83% match1 repo1612024

Rethinking the Role of LLMs in Time Series Forecasting

82% match2026

Large Language Models Are Zero-Shot Time Series Forecasters

82% match2 repos8302026

Forecasting Time Series with LLMs via Patch-Based Prompting and Decomposition

82% match2025

Cite this paper

@article{chernov2024finetuning,
  title  = {Fine-Tuning a Time Series Foundation Model with Wasserstein Loss},
  author = {Andrei Chernov},
  year   = {2024},
  eprint = {2409.15367},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.15367v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.15367.svg)](https://codeofpaper.com/paper/2409.15367)

Discussion