Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights

Jun'ichi Takeuchi, Yoshinari Takeishi, Noboru Murata, Kazushi Mimura, Ka Long Keith Ho +1 more
7/24/2025
cs.LGstat.ML

No confident code implementation found for this paper.

Abstract

Fisher information matrices and neural tangent kernels (NTK) for 2-layer ReLU networks with random hidden weight are argued. We discuss the relation between both notions as a linear transformation and show that spectral decomposition of NTK with concrete forms of eigenfunctions with major eigenvalues. We also obtain an approximation formula of the functions presented by the 2-layer neural networks.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Generalization Ability of Wide Neural Networks on $\mathbb{R}$

82% match2023

Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

81% match2026

Fixing the NTK: From Neural Network Linearizations to Exact Convex Programs

81% match2023

Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regime

81% match2024

An Empirical Analysis of the Laplace and Neural Tangent Kernels

80% match2 repos272022

Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures

80% match1 repo272024

Cite this paper

@article{takeuchi2025neural,
  title  = {Neural Tangent Kernels and Fisher Information Matrices for Simple ReLU Networks with Random Hidden Weights},
  author = {Jun'ichi Takeuchi and Yoshinari Takeishi and Noboru Murata and Kazushi Mimura and Ka Long Keith Ho and Hiroshi Nagaoka},
  year   = {2025},
  eprint = {2507.18555},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2507.18555v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2507.18555.svg)](https://codeofpaper.com/paper/2507.18555)

Discussion