Related papers: The attractors in sequence processing neural netwo…
In this work we investigate the dynamics of random walk processes on scale-free networks in a short to moderate time scale. We perform extensive simulations for the calculation of the mean squared displacement, the network coverage and the…
Considering random noise in finite dimensional parameterized families of diffeomorphisms of a compact finite dimensional boundaryless manifold M, we show the existence of time averages for almost every orbit of each point of M, imposing…
This work is motivated by the relaxation data for materials which exhibit a change of the relationship between the fractional power-law exponents when different relaxation peaks in their dielectric susceptibility are observed. Within the…
In this paper, we are concerned with mean hitting time $\langle\mathcal{H}\rangle$ for random walks on recursive growth tree networks that are built based on an arbitrary tree as the seed via implementing various primitive graphic…
The "edge of chaos" phase transition in artificial neural networks is of renewed interest in light of recent evidence for criticality in brain dynamics. Statistical mechanics traditionally studied this transition with connectivity $k$ as…
The presence of frozen-in or quenched disorder in a system can often modify the nature of its phase transition. A particular instance of this phenomenon is the so-called rounding effect: it has been shown in many cases that the free-energy…
Preferential attachment is a popular model of growing networks. We consider a generalized model with random node removal, and a combination of preferential and random attachment. Using a high-degree expansion of the master equation, we…
The categorization properties of an attractor network of three-state neurons which infers three-state concepts from examples are studied. The evolution equations governing the parallel dynamics at zero temperature for the overlap between…
We present an efficient algorithm for finding all approximate occurrences of a given pattern $p$ of length $m$ in a text $t$ of length $n$ allowing for translocations of equal length adjacent factors and inversions of factors. The algorithm…
We simulate a relaxation process of non-brownian particles in a sheared viscous medium; the small shear strain is initially applied to a system, which then undergoes relaxation. The relaxation time and the correlation length are estimated…
Sequential estimation of a vector of linear regression coefficients is considered under both centralized and decentralized setups. In sequential estimation, the number of observations used for estimation is determined by the observed…
Walking is a fundamental activity of our daily life not only for moving to other places but also for interacting with surrounding environment. While walking on the streets, pedestrians can be aware of attractions like shopping windows. They…
In a recent article [1] we surveyed advances related to adaptation, learning, and optimization over synchronous networks. Various distributed strategies were discussed that enable a collection of networked agents to interact locally in…
Training causal transformers at extreme sequence lengths is bottlenecked by the quadratic time and memory of scaled dot-product attention (SDPA). In this work, we propose Lighthouse Attention, a training-only symmetrical selection-based…
Traditional studies of memory for meaningful narratives focus on specific stories and their semantic structures but do not address common quantitative features of recall across different narratives. We introduce a statistical ensemble of…
Large language models display remarkable capabilities in logical and mathematical reasoning, allowing them to solve complex tasks. Interestingly, these abilities emerge in networks trained on the simple task of next-token prediction. In…
The transformer architecture is central to the success of modern Large Language Models (LLMs), in part due to its surprising ability to perform a wide range of tasks - including mathematical reasoning, memorization, and retrieval - using…
We study the relative entropy and the trace square distance, both of which measure the distance between reduced density matrices of two excited states in two dimensional conformal field theories. We find a general formula for the relative…
The classical preferential attachment model is sensitive to the choice of the initial configuration of the network. As the number of initial nodes and their degree grow, so does the time needed for an equilibrium degree distribution to be…
We study the storage of multiple phase-coded patterns as stable dynamical attractors in recurrent neural networks with sparse connectivity. To determine the synaptic strength of existent connections and store the phase-coded patterns, we…