On the Architectural Complexity of Neural Networks

Nicholas J. Cooper, François G. Meyer, Michael L. Roberts, Carlos Zapata-Carratalá, Lijun Chen +1 more
5/5/2026
cs.LGcs.DMmath.CO

Official code: combinatoriallabs/ArchitecturalComplexity (4 stars, MIT, last commit 2026-05-05).

Abstract

We introduce a unified theoretical framework for the rigorous analysis and systematic construction of deep neural networks (DNNs). This framework addresses a gap in existing theory by explicitly modeling the structure of tensor operations -- lower level information that is often abstracted. Our framework enables two novel objectives: (1) analysis of the evolution of architectural complexity over deep learning history, and (2) automatic construction of novel architectures based on new types of tensor operations. Our study of DNNs introduced over the past 40 years reveals a connection between groundbreaking architectures and increases in different types of architectural complexity. Moreover, we identify several large classes of higher complexity architectures that have not yet been explored. We then collect a dataset of 3,000+ higher complexity architectures, which we publicly release at: https://github.com/combinatoriallabs/ArchitecturalComplexity.

View on arXivView PDF

Code Implementations(1)

combinatoriallabs/ArchitecturalComplexityOfficial100%
MIT

Public-facing repo for code associated to the paper "On the Architectural Complexity of Neural Networks".

41PythonMay 4, 20265 months agoMIT

Similar Papers

Axiomatizing Neural Networks via Pursuit of Subspaces

87% match2026

Structure of Artificial Neural Networks -- Empirical Investigations

87% match2024

Higher Structures in Deep Learning

86% match2026

The Neural Race Reduction: Dynamics of Abstraction in Gated Networks

86% match1 repo82022

Graph Tensor Networks: An Intuitive Framework for Designing Large-Scale Neural Learning Systems on Multiple Domains

84% match2023

Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks

84% match2025

Cite this paper

@article{cooper2026architectural,
  title  = {On the Architectural Complexity of Neural Networks},
  author = {Nicholas J. Cooper and François G. Meyer and Michael L. Roberts and Carlos Zapata-Carratalá and Lijun Chen and Danna Gurari},
  year   = {2026},
  eprint = {2605.04325},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2605.04325v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2605.04325.svg)](https://codeofpaper.com/paper/2605.04325)

Discussion