Related papers: Large Deviations Approach to Random Recurrent Neur…

Neural networks are high-dimensional nonlinear dynamical systems that process information through the coordinated activity of many connected units. Understanding how biological and machine-learning networks function and learn requires…

We establish a way to handle main collective fluctuations in correlated quantum systems based on a Fluctuation Local Field concept. This technique goes beyond standard mean-field approaches, such as Hartree-Fock and dynamical mean-field…

Recurrent neural networks (RNNs) have been used extensively and with increasing success to model various types of sequential data. Much of this progress has been achieved through devising recurrent units and architectures with the…

Recurrent neural networks (RNNs) are brain-inspired models widely used in machine learning for analyzing sequential data. The present work is a contribution towards a deeper understanding of how RNNs process input signals using the response…

We consider artificial neurons which will update their weight coefficients with an internal rule based on backpropagation, rather than using it as an external training procedure. To achieve this we include the backpropagation error estimate…

In this paper, we study the problem of node representation learning with graph neural networks. We present a graph neural network class named recurrent graph neural network (RGNN), that address the shortcomings of prior methods. By using…

In this work, we investigate the use of three information-theoretic quantities -- entropy, mutual information with the class variable, and a class selectivity measure based on Kullback-Leibler divergence -- to understand and study the…

Understanding capabilities and limitations of different network architectures is of fundamental importance to machine learning. Bayesian inference on Gaussian processes has proven to be a viable approach for studying recurrent and deep…

Representations learned by pre-training a neural network on a large dataset are increasingly used successfully to perform a variety of downstream tasks. In this work, we take a closer look at how features are encoded in such pre-trained…

The connectivity of individual neurons of large neural networks determine both the steady state activity of the network and its answer to external stimulus. Highly diluted random networks have zero activity. We show that increasing the…

These notes give a summary of techniques used in large deviation theory to study the fluctuations of time-additive quantities, called dynamical observables, defined in the context of Langevin-type equations, which model equilibrium and…

We study finite-size fluctuations in a network of spiking deterministic neurons coupled with non-uniform synaptic coupling. We generalize a previously developed theory of finite size effects for uniform globally coupled neurons. In the…

One-dimensional run-and-tumble processes may converge towards some localized non-equilibrium steady state when the two velocities and/or the two switching rates are space-dependent. A long dynamical trajectory can be then analyzed via the…

We consider a two-dimensional Hamiltonian system perturbed by a small diffusion term, whose coefficient is state-dependent and non-degenerate. As a result, the process consists of the fast motion along the level curves and slow motion…

We propose a new way of thinking about deep neural networks, in which the linear and non-linear components of the network are naturally derived and justified in terms of principles in probability theory. In particular, the models…

Cortical neurons are characterized by irregular firing and a broad distribution of rates. The balanced state model explains these observations with a cancellation of mean excitatory and inhibitory currents, which makes fluctuations drive…

Anticipation is a strategy used by neural fields to compensate for transmission and processing delays during the tracking of dynamical information, and can be achieved by slow, localized, inhibitory feedback mechanisms such as short-term…

Deep neural networks give us a powerful method to model the training dataset's relationship between input and output. We can regard that as a complex adaptive system consisting of many artificial neurons that work as an adaptive memory as a…

We demonstrate that any Euclidean-time quantum mechanical theory may be represented as a neural network, ensured by the Kosambi-Karhunen-Lo\`eve theorem, mean-square path continuity, and finite two-point functions. The additional constraint…

Several data-driven approaches based on information theory have been proposed for analyzing high-order interactions involving three or more components of a network system. Most of these methods are defined only in the time domain and rely…