相关论文: Asymmetrically extremely dilute neural networks wi…

The parallel dynamics of the asymmetric extremely diluted Ashkin-Teller neural network is studied using signal-to-noise analysis techniques. Evolution equations for the order parameters are derived, both at zero and finite temperature. The…

The parallel dynamics of extremely diluted symmetric Q-Ising neural networks is studied for arbitrary Q using a probabilistic approach. In spite of the extremely diluted architecture the feedback correlations arising from the symmetry…

The microscopic and macroscopic dynamics of random networks is investigated in the strong-dilution limit (i.e. for sparse networks). By simulating chaotic maps, Stuart-Landau oscillators, and leaky integrate-and-fire neurons, we show that a…

We study with numerical simulation the possible limit behaviors of synchronous discrete-time deterministic recurrent neural networks composed of N binary neurons as a function of a network's level of dilution and asymmetry. The network…

We analyze the macroscopic behavior of multi-populations randomly connected neural networks with interaction delays. Similar to cases occurring in spin glasses, we show that the sequences of empirical measures satisfy a large deviation…

Non-reciprocal interactions are a defining feature of many complex systems, biological, ecological, and technological, often pushing them far from equilibrium and enabling rich dynamical responses. These asymmetries can arise at multiple…

We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The…

We study the dynamics of a dilute spherical model with two body interactions and random exchanges. We analyze the Langevin equations and we introduce a functional variational method to study generic dilute disordered models. A crossover…

In natural language processing (NLP), deep neural networks (DNNs) could model complex interactions between context and have achieved impressive results on a range of NLP tasks. Prior works on feature interaction attribution mainly focus on…

Many complex systems operating far from the equilibrium exhibit stochastic dynamics that can be described by a Langevin equation. Inferring Langevin equations from data can reveal how transient dynamics of such systems give rise to their…

We discuss, in this paper, the dynamical properties of extremely diluted, non-monotonic neural networks. Assuming parallel updating and the Hebb prescription for the synaptic connections, a flow equation for the macroscopic overlap is…

Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that…

In spite of the recent interest and advances in linear controllability of complex networks, controlling nonlinear network dynamics remains to be an outstanding problem. We develop an experimentally feasible control framework for nonlinear…

We study the dynamics of on-line learning in large perceptrons, for the case of training sets with a structural bias of the input vectors, by deriving exact and closed macroscopic dynamical laws using non-equilibrium statistical mechanical…

We consider the problem of sampling from a target distribution, which is \emph {not necessarily logconcave}, in the context of empirical risk minimization and stochastic optimization as presented in Raginsky et al. (2017). Non-asymptotic…

Hyperparameter tuning is one of the essential steps to guarantee the convergence of machine learning models. We argue that intuition about the optimal choice of hyperparameters for stochastic gradient descent can be obtained by studying a…

Current deep neural networks are highly overparameterized (up to billions of connection weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient descent algorithms and achieve unexpected levels of…

We develop a unified theory that encompasses the macroscopic dynamics of recurrent interactions of binary units within arbitrary network architectures. Using the martingale theory, our mathematical analysis provides a complete description…

Latent dynamics discovery is challenging in extracting complex dynamics from high-dimensional noisy neural data. Many dimensionality reduction methods have been widely adopted to extract low-dimensional, smooth and time-evolving latent…

In this article we address the problem of the nonlinear interaction of subdiffusive particles. We introduce the random walk model in which statistical characteristics of a random walker such as escape rate and jump distribution depend on…