Related papers: The attractors in sequence processing neural netwo…
We study a neural network model in which both neurons and synaptic interactions evolve in time simultaneously. The time evolution of synaptic interactions is described by a Langevin equation including a Hebbian learning term, and a bias…
We study a simple model in which the growth of a network is determined by the location of one or more random walkers. Depending on walker speed, the model generates a spectrum of structures situated between well-known limiting cases. We…
Complex systems such as protein conformational fluctuations and supercooled liquids exhibit a long relaxation time and are considered to posses multiple relaxation times. We analytically obtain the exact correlation function for stochastic…
For a sample of Exponentially distributed durations we aim at point estimation and a confidence interval for its parameter. A duration is only observed if it has ended within a certain time interval, determined by a Uniform distribution.…
The purpose of this paper is to assess the statistical characterization of weighted networks in terms of the generalization of the relevant parameters, namely average path length, degree distribution and clustering coefficient. Although the…
We clarify the effect different sampling methods and weighting schemes have on the statistics of attractors in ensembles of random Boolean networks (RBNs). We directly measure cycle lengths of attractors and sizes of basins of attraction in…
This paper presents a comprehensive analysis of the degree statistics in models for growing networks where new nodes enter one at a time and attach to one earlier node according to a stochastic rule. The models with uniform attachment,…
A Potts associative memory network has been proposed as a simplified model of macroscopic cortical dynamics, in which each Potts unit stands for a patch of cortex, which can be activated in one of S local attractor states. The internal…
We consider the problem of sequential change detection, where the goal is to design a scheme for detecting any changes in a parameter or functional $\theta$ of the data stream distribution that has small detection delay, but guarantees…
When relaxation towards an equilibrium or steady state is exponential at large times, one usually considers that the associated relaxation time $\tau$, i.e., the inverse of that decay rate, is the longest characteristic time in the system.…
Transformer architectures have led to remarkable progress in many state-of-art applications. However, despite their successes, modern transformers rely on the self-attention mechanism, whose time- and space-complexity is quadratic in the…
Despite the remarkable empirical performance of Transformers, their theoretical understanding remains elusive. Here, we consider a deep multi-head self-attention network, that is closely related to Transformers yet analytically tractable.…
We propose a general method for the construction and analysis of unweighted $\epsilon$ - recurrence networks from chaotic time series. The selection of the critical threshold $\epsilon_c$ in our scheme is done empirically and we show that…
While criticality is widely observed in neural networks, its underlying neural mechanism is not known well. We consider a network of $N$ excitatory leaky integrated and fire (LIF) neurons that reside on a regular lattice with periodic…
The Transformer architecture is widely applied in sequence modeling applications, yet the theoretical understanding of its working principles remains limited. In this work, we investigate the approximation rate for single-layer Transformers…
For a given base class of sequence-to-next-token generators, we consider learning prompt-to-answer mappings obtained by iterating a fixed, time-invariant generator for multiple steps, thus generating a chain-of-thought, and then taking the…
This paper presents a simple periodic parameter-switching method which can find any stable limit cycle that can be numerically approximated in a generalized Duffing system. In this method, the initial value problem of the system is…
In this work, we extend standard neural networks by building upon an assumption that neuronal activations correspond to the angle of a complex number lying on the unit circle, or 'phasor.' Each layer in such a network produces new…
Contraction analysis considers the distance between two adjacent trajectories. If this distance is contracting, then trajectories have the same long-term behavior. The main advantage of this analysis is that it is independent of the…
In this paper, we propose a general framework for the trapping problem on a weighted network with a perfect trap fixed at an arbitrary node. By utilizing the spectral graph theory, we provide an exact formula for mean first-passage time…