Metric entropy of causal, discrete-time LTI systems
Dynamical Systems
2022-11-29 v1 Information Theory
math.IT
Abstract
In [1] it is shown that recurrent neural networks (RNNs) can learn - in a metric entropy optimal manner - discrete time, linear time-invariant (LTI) systems. This is effected by comparing the number of bits needed to encode the approximating RNN to the metric entropy of the class of LTI systems under consideration [2, 3]. The purpose of this note is to provide an elementary self-contained proof of the metric entropy results in [2, 3], in the process of which minor mathematical issues appearing in [2, 3] are cleaned up. These corrections also lead to the correction of a constant in a result in [1] (see Remark 2.5).
Cite
@article{arxiv.2211.15466,
title = {Metric entropy of causal, discrete-time LTI systems},
author = {Clemens Hutter and Thomas Allard and Helmut Bölcskei},
journal= {arXiv preprint arXiv:2211.15466},
year = {2022}
}
Comments
[1] arXiv:2105.02556