Related papers: The macroscopic dynamics in separable neural netwo…

Over the past two decades, there has been a tremendous increase in the growth of representation learning methods for graphs, with numerous applications across various fields, including bioinformatics, chemistry, and the social sciences.…

We introduce a notion of emergence for coarse-grained macroscopic variables associated with highly-multivariate microscopic dynamical processes, in the context of a coupled dynamical environment. Dynamical independence instantiates the…

This work develops the global equations of neural networks through stacked piecewise manifolds, fixed-point theory, and boundary-conditioned iteration. Once fixed coordinates and operators are removed, a neural network appears as a…

We propose a new approach to constructing a neural network for predicting expectations of stochastic differential equations. The proposed method does not need data sets of inputs and outputs; instead, the information obtained from the…

Several recent trends in machine learning theory and practice, from the design of state-of-the-art Gaussian Process to the convergence analysis of deep neural nets (DNNs) under stochastic gradient descent (SGD), have found it fruitful to…

We present a structured neural network architecture that is inspired by linear time-varying dynamical systems. The network is designed to mimic the properties of linear dynamical systems which makes analysis and control simple. The…

We derive an exact representation of the topological effect on the dynamics of sequence processing neural networks within signal-to-noise analysis. A new network structure parameter, loopiness coefficient, is introduced to quantitatively…

Network topology plays an important role in governing the collective dynamics. Partial synchronization (PaS) on regular networks with a few non-local links is explored. Different PaS patterns out of the symmetry breaking are observed for…

Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions. Computational models of sequence memory have been proposed where…

Harnessing parallelism in seemingly sequential models is a central challenge for modern machine learning. Several approaches have been proposed for evaluating sequential processes in parallel using iterative fixed-point methods, like…

In sustained growth with random dynamics stationary distributions can exist without detailed balance. This suggests thermodynamical behavior in fast growing complex systems. In order to model such phenomena we apply both a discrete and a…

Developments in dynamical systems theory provides new support for the macroscale modelling of pdes and other microscale systems such as Lattice Boltzmann, Monte Carlo or Molecular Dynamics simulators. By systematically resolving subgrid…

We compute the joint large deviation rate functional in the limit of large time for the current flowing through the edges of a finite graph on which a boundary-driven system of stochastic particles evolves with zero-range dynamics.This…

Performance of standard processes over large distributed networks typically scales with the size of the network. For example, in planar topologies where nodes communicate with their natural neighbors, the scaling factor is $O(n)$, where $n$…

Vanilla spiking neurons in Spiking Neural Networks (SNNs) use charge-fire-reset neuronal dynamics, which can only be simulated serially and can hardly learn long-time dependencies. We find that when removing reset, the neuronal dynamics can…

A fundamental problem in neuroscience is to characterize the dynamics of spiking from the neurons in a circuit that is involved in learning about a stimulus or a contingency. A key limitation of current methods to analyze neural spiking…

Network Markov Decision Processes (MDPs), a popular model for multi-agent control, pose a significant challenge to efficient learning due to the exponential growth of the global state-action space with the number of agents. In this work,…

Time evolution of diluted neural networks with a nonmonotonic transfer function is analitically described by flow equations for macroscopic variables. The macroscopic dynamics shows a rich variety of behaviours: fixed-point, periodicity and…

A stochastic differential equation that describes the dynamics of single-domain magnetic particles at any temperature is derived using a classical formalism. The deterministic terms recover existing theory and the stochastic process takes…

In this article, we present an efficient deep learning method called coupled deep neural networks (CDNNs) for coupled physical problems. Our method compiles the interface conditions of the coupled PDEs into the networks properly and can be…