Related papers: Large Deviations Approach to Random Recurrent Neur…
We prove a large deviation principle for deep neural networks with Gaussian weights and at most linearly growing activation functions, such as ReLU. This generalises earlier work, in which bounded and continuous activation functions were…
Vanishing (and exploding) gradients effect is a common problem for recurrent neural networks with nonlinear activation functions which use backpropagation method for calculation of derivatives. Deep feedforward neural networks with many…
This paper studies probabilistic rates of convergence for consensus+innovations type of algorithms in random, generic networks. For each node, we find a lower and also a family of upper bounds on the large deviations rate function, thus…
We study two-layer belief networks of binary random variables in which the conditional probabilities Pr[childlparents] depend monotonically on weighted sums of the parents. In large networks where exact probabilistic inference is…
Long-range correlated processes are ubiquitous, ranging from climate variables to financial time series. One paradigmatic example for such processes is fractional Brownian motion (fBm). In this work, we highlight the potentials and…
We consider the effects of long-range temporal correlations in many-particle systems, focusing particularly on fluctuations about the typical behaviour. For a specific class of memory dependence we discuss the modification of the large…
We consider a network of randomly coupled rate-based neurons influenced by external and internal noise. We derive a second-order stochastic mean-field model for the network dynamics and use it to analyze the stability and bifurcations in…
In active Brownian motion, an internal propulsion mechanism interacts with translational and rotational thermal noise and other internal fluctuations to produce directed motion. We derive the distribution of its extreme fluctuations and…
Networked dynamical systems, i.e., systems of dynamical units coupled via nontrivial interaction topologies, constitute models of broad classes of complex systems, ranging from gene regulatory and metabolic circuits in our cells to…
How the information microscopically processed by individual neurons is integrated and used in organizing the behavior of an animal is a central question in neuroscience. The coherence of neuronal dynamics over different scales has been…
Stochastic reaction-diffusion models can be analytically studied on complex networks using the linear noise approximation. This is illustrated through the use of a specific stochastic model, which displays traveling waves in its…
In this short note we consider semi-Markov processes satisfying the condition of direction-time independence (Markov renewal processes). We derive large deviation principles and fluctuation theorems for the empirical current and the…
We propose a novel approach for inferring the individualized causal effects of a treatment (intervention) from observational data. Our approach conceptualizes causal inference as a multitask learning problem; we model a subject's potential…
Reinforcement learning (RL) is well known for requiring large amounts of data in order for RL agents to learn to perform complex tasks. Recent progress in model-based RL allows agents to be much more data-efficient, as it enables them to…
Massively parallel recordings of spiking activity in cortical networks show that covariances vary widely across pairs of neurons. Their low average is well understood, but an explanation for the wide distribution in relation to the static…
These lecture notes develop the theory of learning in deep and recurrent neuronal networks from the point of view of Bayesian inference. The aim is to enable the reader to understand typical computations found in the literature in this…
A key attribute that drives the unprecedented success of modern Recurrent Neural Networks (RNNs) on learning tasks which involve sequential data, is their ability to model intricate long-term temporal dependencies. However, a well…
We discuss the effects of common synaptic inputs in a recurrent neural network. Because of the effects of these common synaptic inputs, the correlation between neural inputs cannot be ignored, and thus the network exhibits sample…
Deep neural networks (DNNs) have achieved remarkable empirical success, yet their training dynamics remain understood mainly from optimization rather than statistical principles. Here we develop a statistical framework for DNN training in…
Randomized neural networks (NNs) are an interesting alternative to conventional NNs that are more used for data modeling. The random vector functional-link (RVFL) network is an established and theoretically well-grounded randomized learning…