Shared Representational Geometry Across Neural Networks

Qihong Lu, Po-Hsuan Chen, Jonathan W. Pillow, Peter J. Ramadge, Kenneth A. Norman +1 more
11/28/2018
cs.LGstat.ML

Official code: qihongl/nnsrm-neurips18 (8 stars, MIT, TensorFlow, last commit 2019-03-22).

Abstract

Different neural networks trained on the same dataset often learn similar input-output mappings with very different weights. Is there some correspondence between these neural network solutions? For linear networks, it has been shown that different instances of the same network architecture encode the same representational similarity matrix, and their neural activity patterns are connected by orthogonal transformations. However, it is unclear if this holds for non-linear networks. Using a shared response model, we show that different neural networks encode the same input examples as different orthogonal transformations of an underlying shared representation. We test this claim using both standard convolutional neural networks and residual networks on CIFAR10 and CIFAR100.

View on arXivView PDF

Code Implementations(1)

qihongl/nnsrm-neurips18Official100%
TensorFlowMIT

Lu, et al. (2018) Shared Representational Geometry Across Neural Networks. Workshop on Integration of Deep Learning Theories, NeurIPS.

80Python, Jupyter NotebookMar 13, 20197 years agoMIT
deep-learningfunctional-alignmentkerasneural-networksrepresentation

Similar Papers

Similarity of Neural Networks with Gradients

85% match1 repo2572020

Insights on representational similarity in neural networks with canonical correlation

84% match1 repo6442018

Representational Similarity via Interpretable Visual Concepts

84% match1 repo72025

Comparing and Integrating Different Notions of Representational Correspondence in Neural Systems

83% match2025

Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks

83% match2026

Similarity of Neural Network Models: A Survey of Functional and Representational Measures

82% match1 repo42023

Cite this paper

@article{lu2018shared,
  title  = {Shared Representational Geometry Across Neural Networks},
  author = {Qihong Lu and Po-Hsuan Chen and Jonathan W. Pillow and Peter J. Ramadge and Kenneth A. Norman and Uri Hasson},
  year   = {2018},
  eprint = {1811.11684},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1811.11684}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1811.11684.svg)](https://codeofpaper.com/paper/1811.11684)

Discussion