Paths and Ambient Spaces in Neural Loss Landscapes

Daniel Dold, Julius Kobialka, Nicolai Palm, Emanuel Sommer, David Rügamer +1 more
3/5/2025
cs.LGstat.ML

No official code found. Best implementation: doldd/Paths-and-Ambient-Spaces (2 stars, MIT, last commit 2025-04-14).

Abstract

Understanding the structure of neural network loss surfaces, particularly the emergence of low-loss tunnels, is critical for advancing neural network theory and practice. In this paper, we propose a novel approach to directly embed loss tunnels into the loss landscape of neural networks. Exploring the properties of these loss tunnels offers new insights into their length and structure and sheds light on some common misconceptions. We then apply our approach to Bayesian neural networks, where we improve subspace inference by identifying pitfalls and proposing a more natural prior that better guides the sampling procedure.

View on arXivView PDF

Code Implementations(1)

doldd/Paths-and-Ambient-SpacesCommunity100%
MIT
20R, PythonMar 4, 20251 years agoMIT

Similar Papers

Evaluating Loss Landscapes from a Topology Perspective

86% match2024

Input Space Mode Connectivity in Deep Neural Networks

83% match2024

Visualizing the loss landscapes of physics-informed neural networks

83% match2026

A topological description of loss surfaces based on Betti Numbers

83% match2024

Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes

82% match1 repo32024

Flat Channels to Infinity in Neural Loss Landscapes

82% match2025

Cite this paper

@article{dold2025paths,
  title  = {Paths and Ambient Spaces in Neural Loss Landscapes},
  author = {Daniel Dold and Julius Kobialka and Nicolai Palm and Emanuel Sommer and David Rügamer and Oliver Dürr},
  year   = {2025},
  eprint = {2503.03382},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2503.03382v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.03382.svg)](https://codeofpaper.com/paper/2503.03382)

Discussion