Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery

Gilad Lerman, Kang Li, Tyler Maunu, Teng Zhang
6/25/2025
stat.MLcs.LGmath.OC

No official code found. Best implementation: alp-del/FMS_simulations (0 stars, MIT, last commit 2026-03-06).

Abstract

Robust subspace estimation is fundamental to many machine learning and data analysis tasks. Iteratively Reweighted Least Squares (IRLS) is an elegant and empirically effective approach to this problem, yet its theoretical properties remain poorly understood. This paper establishes that, under deterministic conditions, a variant of IRLS with dynamic smoothing regularization converges linearly to the underlying subspace from any initialization. We extend these guarantees to affine subspace estimation, a setting that lacks prior recovery theory. Additionally, we illustrate the practical benefits of IRLS through an application to low-dimensional neural network training. Our results provide the first global convergence guarantees for IRLS in robust subspace recovery and, more broadly, for nonconvex IRLS on a Riemannian manifold.

View on arXivView PDF

Code Implementations(1)

alp-del/FMS_simulationsCommunity100%
MIT
00Shell, PythonDec 1, 20247 months agoMIT

Similar Papers

Precise asymptotics of reweighted least-squares algorithms for linear diagonal networks

82% match2024

A universal linearized subspace refinement framework for neural networks

78% match2026

Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay

77% match2025

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis

77% match2024

A Scalable Approach for Safe and Robust Learning via Lipschitz-Constrained Networks

77% match2025

Gradient Descent in Neural Networks as Sequential Learning in RKBS

76% match2023

Cite this paper

@article{lerman2025global,
  title  = {Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery},
  author = {Gilad Lerman and Kang Li and Tyler Maunu and Teng Zhang},
  year   = {2025},
  eprint = {2506.20533},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2506.20533v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2506.20533.svg)](https://codeofpaper.com/paper/2506.20533)

Discussion