Iteratively Reweighted $\ell_1$-Penalized Robust Regression

Xiaoou Pan, Qiang Sun, Wen-Xin Zhou
7/9/2019
math.STstat.ML

Official code: XiaoouPan/ILAMM (13 stars, last commit 2022-05-09).

Abstract

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moment, we show that iteratively reweighted $\ell_1$-penalized adaptive Huber regression estimator satisfies exponential deviation bounds and oracle properties, including the oracle convergence rate and variable selection consistency, under a weak beta-min condition. Computationally, we need as many as $O(\log s + \log\log d)$ iterations to reach such an oracle estimator, where $s$ and $d$ denote the sparsity and ambient dimension, respectively. Extension to a general class of robust loss functions is also considered. Numerical studies lend strong support to our methodology and theory.

View on arXivView PDF

Code Implementations(1)

XiaoouPan/ILAMMOfficial100%

Nonconvex Regularized Robust Regression via I-LAMM Algorithm

135R, C++Dec 12, 20184 years ago

Similar Papers

The Adaptive τ-Lasso: Robustness and Oracle Properties

83% match2023

The Theory and Practice of Highly Scalable Gaussian Process Regression with Nearest Neighbours

80% match2026

Understanding Robust Machine Learning for Nonparametric Regression with Heavy-Tailed Noise

80% match2025

Statistical inverse learning and $\ell^1$-regularization

80% match2026

Exponential Lasso: robust sparse penalization under heavy-tailed noise and outliers with exponential-type loss

80% match1 repo2025

High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality

79% match2023

Cite this paper

@article{pan2019iteratively,
  title  = {Iteratively Reweighted $\ell_1$-Penalized Robust Regression},
  author = {Xiaoou Pan and Qiang Sun and Wen-Xin Zhou},
  year   = {2019},
  eprint = {1907.04027},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1907.04027}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1907.04027.svg)](https://codeofpaper.com/paper/1907.04027)

Discussion