Robust variance-regularized risk minimization with concomitant scaling

Matthew J. Holland
1/27/2023
stat.MLcs.LG

No official code found. Best implementation: feedbackward/bdd-mv (0 stars, MIT, last commit 2025-03-06).

Abstract

Under losses which are potentially heavy-tailed, we consider the task of minimizing sums of the loss mean and standard deviation, without trying to accurately estimate the variance. By modifying a technique for variance-free robust mean estimation to fit our problem setting, we derive a simple learning procedure which can be easily combined with standard gradient-based solvers to be used in traditional machine learning workflows. Empirically, we verify that our proposed approach, despite its simplicity, performs as well or better than even the best-performing candidates derived from alternative criteria such as CVaR or DRO risks on a variety of datasets.

View on arXivView PDF

Code Implementations(1)

feedbackward/bdd-mvCommunity100%
MIT

bdd-mv: mean-variance minimization using bi-directional dispersion functions

00Shell, PythonJan 27, 20231 years agoMIT

Similar Papers

Empirical Risk Minimization for Losses without Variance

81% match2023

Distributionally Robust Optimization with Bias and Variance Reduction

80% match1 repo82023

Understanding Robust Machine Learning for Nonparametric Regression with Heavy-Tailed Noise

78% match2025

A Simpler Alternative to Variational Regularized Counterfactual Risk Minimization

78% match2024

Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificity

78% match1 repo22023

Functional Risk Minimization

77% match2024

Cite this paper

@article{holland2023robust,
  title  = {Robust variance-regularized risk minimization with concomitant scaling},
  author = {Matthew J. Holland},
  year   = {2023},
  eprint = {2301.11584},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.11584v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.11584.svg)](https://codeofpaper.com/paper/2301.11584)

Discussion