Related papers: Thermodynamic limit in learning period three

The learning dynamics of deep neural networks are not well understood. The information bottleneck (IB) theory proclaimed separate fitting and compression phases. But they have since been heavily debated. We comprehensively analyze the…

Deep learning models evolve through training to learn the manifold in which the data exists to satisfy an objective. It is well known that evolution leads to different final states which produce inconsistent predictions of the same test…

In recent years, nonlinear dynamic system identification using artificial neural networks has garnered attention due to its broad potential applications across science and engineering. However, purely data-driven approaches often struggle…

In the realm of fractal geometry, intricate structures emerge from simple iterative processes that partition parameter spaces into regions of stability and instability. Likewise, training large language models involves iteratively applying…

As deep neural networks grow in size, from thousands to millions to billions of weights, the performance of those networks becomes limited by our ability to accurately train them. A common naive question arises: if we have a system with…

We analyze how the transient dynamics of large dynamical systems in the vicinity of a stationary point, modeled by a set of randomly coupled linear differential equations, depends on the network topology. We characterize the transient…

Nonlinear normal modes are periodic orbits that survive in nonlinear many-body Hamiltonian systems, and their instability is crucial for relaxation dynamics. Here, we study the instability process of the $\pi/3$-mode in the…

In the classical herding model, asymptotic learning refers to situations where individuals eventually take the correct action regardless of their private information. Classical results identify classes of information structures for which…

Transitivity, the existence of periodic points and positive topological entropy can be used to characterize complexity in dynamical systems. It is known that for graphs that are not trees, for every $\varepsilon>0,$ there exist (complicate)…

The capability of recurrent neural networks to approximate trajectories of a random dynamical system, with random inputs, on non-compact domains, and over an indefinite or infinite time horizon is considered. The main result states that…

We introduce probabilistic neural networks that describe unsupervised synchronous learning on an atomic Hardy space and space of bounded real analytic functions, respectively. For a stationary ergodic vector process, we prove that the…

Representation learning from complex data typically involves models with a large number of parameters, which in turn require large amounts of data samples. In neural network models, model complexity grows with the number of inputs to each…

Thermodynamics-informed neural networks employ inductive biases for the enforcement of the first and second principles of thermodynamics. To construct these biases, a metriplectic evolution of the system is assumed. This provides excellent…

The fundamental learning theory behind neural networks remains largely open. What classes of functions can neural networks actually learn? Why doesn't the trained network overfit when it is overparameterized? In this work, we prove that…

We study bifurcations of area-preserving maps, both orientable (symplectic) and non-orientable, with quadratic homoclinic tangencies. We consider one and two parameter general unfoldings and establish results related to the appearance of…

We study the dynamics of discrete-time regulatory networks on random digraphs. For this we define ensembles of deterministic orbits of random regulatory networks, and introduce some statistical indicators related to the long-term dynamics…

In this work, we consider a class of $n$-dimensional, $n\geq2$, piecewise linear discontinuous maps that can exhibit a new type of attractor, called a weird quasiperiodic attractor. While the dynamics associated with these attractors may…

We demonstrate, both analytically and numerically, that learning dynamics of neural networks is generically attracted towards a self-organized critical state. The effect can be modeled with quartic interactions between non-trainable…

We discuss the phase transition and critical exponents in the random allocation model (urn model) for different statistical ensembles. We provide a unified presentation of the statistical properties of the model in the thermodynamic limit,…

We extend Sharkovskii's theorem to the cases of $N$-dimensional maps which are close to 1D maps, with an attracting $n$-periodic orbit. We prove that, with relatively weak topological assumptions, there exist also $m$-periodic orbits for…