The Galerkin method beats Graph-Based Approaches for Spectral Algorithms

Vivien Cabannes, Francis Bach
6/1/2023
cs.LGcs.AIstat.ML

No official code found. Best implementation: VivienCabannes/laplacian (17 stars, MIT, last commit 2024-05-31).

Abstract

Historically, the machine learning community has derived spectral decompositions from graph-based approaches. We break with this approach and prove the statistical and computational superiority of the Galerkin method, which consists in restricting the study to a small set of test functions. In particular, we introduce implementation tricks to deal with differential operators in large dimensions with structured kernels. Finally, we extend on the core principles beyond our approach to apply them to non-linear spaces of functions, such as the ones parameterized by deep neural networks, through loss-based optimization procedures.

View on arXivView PDF

Code Implementations(1)

VivienCabannes/laplacianCommunity100%
MIT

Fast Laplacian estimation

172Shell, PythonOct 10, 20222 years agoMIT

Similar Papers

Spectral Neural Networks: Approximation Theory and Optimization Landscape

84% match2023

Neural Spectral Methods: Self-supervised learning in the spectral domain

80% match1 repo482026

Approaching Deep Learning through the Spectral Dynamics of Weights

80% match1 repo132024

Asymmetric Learning for Spectral Graph Neural Networks

79% match1 repo22024

When do spectral gradient updates help in deep learning?

79% match1 repo182025

Optimization for Neural Operators can Benefit from Width

79% match2025

Cite this paper

@article{cabannes2023galerkin,
  title  = {The Galerkin method beats Graph-Based Approaches for Spectral Algorithms},
  author = {Vivien Cabannes and Francis Bach},
  year   = {2023},
  eprint = {2306.00742},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.00742v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.00742.svg)](https://codeofpaper.com/paper/2306.00742)

Discussion