Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks

Sahel Vahedi Noori, Bin Hu, Geir Dullerud, Peter Seiler
5/8/2024
eess.SYcs.LGmath.OC

No confident code implementation found for this paper.

Abstract

This paper presents sufficient conditions for the stability and $\ell_2$-gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

A Complete Set of Quadratic Constraints for Repeated ReLU and Generalizations

86% match2024

Generative System Dynamics in Recurrent Neural Networks

82% match2025

Testing Stationarity Concepts for ReLU Networks: Hardness, Regularity, and Robust Algorithms

81% match2023

Unconditional stability of a recurrent neural circuit implementing divisive normalization

80% match1 repo72024

Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks

79% match2023

Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks

79% match2026

Cite this paper

@article{noori2024stability,
  title  = {Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks},
  author = {Sahel Vahedi Noori and Bin Hu and Geir Dullerud and Peter Seiler},
  year   = {2024},
  eprint = {2405.05236},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.05236v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.05236.svg)](https://codeofpaper.com/paper/2405.05236)

Discussion