Understand the Effect of Importance Weighting in Deep Learning on Dataset Shift

Thien Nhan Vo
5/6/2025
cs.LG

No official code found. Best implementation: thnhan05/Understand-the-Effect-of-Importance-Weighting-in-Deep-Learning-on-Dataset-Shift (0 stars, last commit 2025-07-10).

Abstract

We evaluate the effectiveness of importance weighting in deep neural networks under label shift and covariate shift. On synthetic 2D data (linearly separable and moon-shaped) using logistic regression and MLPs, we observe that weighting strongly affects decision boundaries early in training but fades with prolonged optimization. On CIFAR-10 with various class imbalances, only L2 regularization (not dropout) helps preserve weighting effects. In a covariate-shift experiment, importance weighting yields no significant performance gain, highlighting challenges on complex data. Our results call into question the practical utility of importance weighting for real-world distribution shifts.

View on arXivView PDF

Code Implementations(1)

thnhan05/Understand-the-Effect-of-Importance-Weighting-in-Deep-Learning-on-Dataset-ShiftCommunity70%

Code paper "Understanding the Effect of Importance Weighting in Deep Learning under Dataset Shift"

00Python, Jupyter NotebookJul 10, 20251 years ago

Similar Papers

A Short Survey on Importance Weighting for Machine Learning

84% match2024

When is Importance Weighting Correction Needed for Covariate Shift Adaptation?

84% match2023

Why Loss Re-weighting Works If You Stop Early: Training Dynamics of Unconstrained Features

84% match2026

Improving importance estimation in covariate shift for providing accurate prediction error

81% match2024

Importance Sampling for Stochastic Gradient Descent in Deep Neural Networks

80% match2023

Simplifying Neural Network Training Under Class Imbalance

79% match1 repo92026

Cite this paper

@article{vo2025understand,
  title  = {Understand the Effect of Importance Weighting in Deep Learning on Dataset Shift},
  author = {Thien Nhan Vo},
  year   = {2025},
  eprint = {2505.03617},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2505.03617v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2505.03617.svg)](https://codeofpaper.com/paper/2505.03617)

Discussion