Related papers: Statistical Neurodynamics for sequence processing …
We present a new theoretical perspective of data noising in recurrent neural network language models (Xie et al., 2017). We show that each variant of data noising is an instance of Bayesian recurrent neural networks with a particular…
Signal transduction networks can form highly interconnected systems within cells due to network crosstalk, the sharing of input signals between multiple downstream responses. To better understand the evolutionary design principles…
We analyze the effects of noise on the permutation entropy of dynamical systems. We take as numerical examples the logistic map and the R\"ossler system. Upon varying the noise strengthfaster, we find a transition from an…
Noise Contrastive Estimation (NCE) is a popular approach for learning probability density functions parameterized up to a constant of proportionality. The main idea is to design a classification problem for distinguishing training data from…
Generative diffusion processes are an emerging and effective tool for image and speech generation. In the existing methods, the underline noise distribution of the diffusion process is Gaussian noise. However, fitting distributions with…
This article studies typical dynamics and fluctuations for a slow-fast dynamical system perturbed by a small fractional Brownian noise. Based on an ergodic theorem with explicit rates of convergence, which may be of independent interest, we…
It is well known that chaotic dynamic systems (such as three-body system, turbulent flow and so on) have the sensitive dependance on initial conditions (SDIC). Unfortunately, numerical noises (such as truncation error and round-off error)…
Neural-network models of high-level brain functions such as memory recall and reasoning often rely on the presence of stochasticity. The majority of these models assumes that each neuron in the functional network is equipped with its own…
In this paper, we study a class of equations representing nonlinear diffusion on networks. A particular instance of our model can be seen as a network equivalent of the porous-medium equation. We are interested in studying perturbations of…
Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing. Their nonsmooth counterparts, modified Gauss-Newton or prox-linear algorithms, can lead to contrasting outcomes when compared…
We consider statistical inference for a class of mixed-effects models with system noise described by a non-Gaussian integrated Ornstein-Uhlenbeck process. Under the asymptotics where the number of individuals goes to infinity with possibly…
Natural gradient descent is a principled method for adapting the parameters of a statistical model on-line using an underlying Riemannian parameter space to redefine the direction of steepest descent. The algorithm is examined via methods…
Learning a parametric model of a data distribution is a well-known statistical problem that has seen renewed interest as it is brought to scale in deep learning. Framing the problem as a self-supervised task, where data samples are…
Second-order phase transitions are characterised by critical scaling and universality. The singular behaviour of thermodynamic quantities at the transition, in particular, is determined by critical exponents of the universality class of the…
We study properties of the cross-sectional distribution of returns. A significant anti-correlation between dispersion and cross-sectional kurtosis is found such that dispersion is high but kurtosis is low in panic times, and the opposite in…
A multiscale analysis of 1D stochastic bistable reaction-diffusion equations with additive noise is carried out w.r.t. travelling waves within the variational approach to stochastic partial differential equations. It is shown with explicit…
We investigate the efficient transmission and processing of weak, subthreshold signals in a realistic neural medium in the presence of different levels of the underlying noise. Assuming Hebbian weights for maximal synaptic conductances --…
Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods that assume consistent intervals and complete data. Neural Ordinary Differential Equations (Neural ODEs) offer an…
This article studies the dynamics of the mean-field approximation of continuous random networks. These networks are stochastic integrodifferential equations driven by Gaussian noise. The kernels in the integral operators are realizations of…
Performing gradient descent in a wide neural network is equivalent to computing the posterior mean of a Gaussian Process with the Neural Tangent Kernel (NTK-GP), for a specific prior mean and with zero observation noise. However, existing…