相关论文: Fading memory echo state networks are universal

We give a universal kernel that renders all the regular languages linearly separable. We are not able to compute this kernel efficiently and conjecture that it is intractable, but we do have an efficient $\eps$-approximation.

We define a neural network in infinite dimensional spaces for which we can show the universal approximation property. Indeed, we derive approximation results for continuous functions from a Fr\'echet space $\X$ into a Banach space $\Y$. The…

Recent advancements in artificial neural networks have enabled impressive tasks on classical computers, but they demand significant computational resources. While quantum computing offers potential beyond classical systems, the advantages…

The universality of a quantum neural network refers to its ability to approximate arbitrary functions and is a theoretical guarantee for its effectiveness. A non-universal neural network could fail in completing the machine learning task.…

Spiking neural networks (SNNs) are recurrent models that can leverage sparsity in input time series to efficiently carry out tasks such as classification. Additional efficiency gains can be obtained if decisions are taken as early as…

Approximation capability of reservoir systems whose reservoir is a recurrent neural network (RNN) is discussed. We show what we call uniform strong universality of RNN reservoir systems for a certain class of dynamical systems. This means…

Associative memory models are content-addressable memory systems fundamental to biological intelligence and are notable for their high interpretability. However, existing models evaluate the quality of retrieval based on proximity, which…

We derive fundamental lower bounds on the connectivity and the memory requirements of deep neural networks guaranteeing uniform approximation rates for arbitrary function classes in $L^2(\mathbb R^d)$. In other words, we establish a…

This work advances the theoretical foundations of reservoir computing (RC) by providing a unified treatment of fading memory and the echo state property (ESP) in both deterministic and stochastic settings. We investigate state-space…

The network studied here is based on a standard model in physics, but it appears in various applications ranging from spintronics to neuroscience. When the network is forced by an external signal common to all its elements, there are shown…

In this paper we consider switched nonlinear systems under average dwell time switching signals, with an otherwise arbitrary compact index set and with additional constraints in the switchings. We present invariance principles for these…

In this paper, we establish universal approximation theorems for neural networks applied to general nonlinear ill-posed operator equations. In addition to the approximation error, the measurement error is also taken into account in our…

The decay properties of the semigroup generated by a linear Timoshenko system with fading memory are discussed. Uniform stability is shown to occur within a necessary and sufficient condition on the memory kernel.

Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines. Recent ``deep-learning-style'' implementations of PCs strive for a better scalability,…

We propose Echo State Networks (ESNs) to predict the statistics of extreme events in a turbulent flow. We train the ESNs on small datasets that lack information about the extreme events. We asses whether the networks are able to extrapolate…

Echo State Networks are efficient time-series predictors, which highly depend on the value of the spectral radius of the reservoir connectivity matrix. Based on recent results on the mean field theory of driven random recurrent neural…

Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. This approach…

While Large Language Models and their underlying Transformer architecture are remarkably efficient, they do not reflect how our brain processes and learns a diversity of cognitive tasks such as language, nor how it leverages working memory.…

The paper is a follow-up of the recently introduced kernel-based framework to identify nonlinear input-output systems regularized by desirable input-output incremental properties. Assuming that the system has fading memory, we propose to…

The capacity of non-coherent stationary Gaussian fading channels with memory under a peak-power constraint is studied in the asymptotic weak-signal regime. It is assumed that the fading law is known to both transmitter and receiver but that…