Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks
Systems and Control
2024-05-15 v3 Machine Learning
Systems and Control
Optimization and Control
Abstract
This paper presents sufficient conditions for the stability and -gain performance of recurrent neural networks (RNNs) with ReLU activation functions. These conditions are derived by combining Lyapunov/dissipativity theory with Quadratic Constraints (QCs) satisfied by repeated ReLUs. We write a general class of QCs for repeated RELUs using known properties for the scalar ReLU. Our stability and performance condition uses these QCs along with a "lifted" representation for the ReLU RNN. We show that the positive homogeneity property satisfied by a scalar ReLU does not expand the class of QCs for the repeated ReLU. We present examples to demonstrate the stability / performance condition and study the effect of the lifting horizon.
Keywords
Cite
@article{arxiv.2405.05236,
title = {Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks},
author = {Sahel Vahedi Noori and Bin Hu and Geir Dullerud and Peter Seiler},
journal= {arXiv preprint arXiv:2405.05236},
year = {2024}
}