Related papers: Revisiting Deep Information Propagation: Fractal F…
Quantum spin networks overcome the challenges of traditional charge-based electronics by encoding the information into spin degrees of freedom. Although beneficial for transmitting information with minimal losses when compared to their…
We study the behavior of untrained neural networks whose weights and biases are randomly distributed using mean field theory. We show the existence of depth scales that naturally limit the maximum depth of signal propagation through these…
We introduce a design strategy for neural network macro-architecture based on self-similarity. Repeated application of a simple expansion rule generates deep networks whose structural layouts are precisely truncated fractals. These networks…
We propose a mathematical framework to systematically explore the propagation properties of a class of continuous in time nonlinear neural network models comprising a hierarchy of processing areas, mutually connected according to the…
We introduce the concept of boundaries of a complex network as the set of nodes at distance larger than the mean distance from a given node in the network. We study the statistical properties of the boundaries nodes of complex networks. We…
Assessing where and how information is stored in biological networks (such as neuronal and genetic networks) is a central task both in neuroscience and in molecular genetics, but most available tools focus on the network's structure as…
Feed-forward deep neural networks have been used extensively in various machine learning applications. Developing a precise understanding of the underling behavior of neural networks is crucial for their efficient deployment. In this paper,…
Complex systems with multiple processes evolving on different temporal scales are naturally described by multilayer networks, where each layer represents a different timescale. In this work, we show how the multilayer structure shapes the…
In the realm of fractal geometry, intricate structures emerge from simple iterative processes that partition parameter spaces into regions of stability and instability. Likewise, training large language models involves iteratively applying…
We quantify the finite size effects in a stochastic network made up of rate neurons, for several kinds of recurrent connectivity matrices. This analysis is performed by means of a perturbative expansion of the neural equations, where the…
We analyze the dynamics of finite width effects in wide but finite feature learning neural networks. Starting from a dynamical mean field theory description of infinite width deep neural network kernel and prediction dynamics, we provide a…
The mechanisms of information transmission are investigated in a lattice of coupled continuous maps, by analyzing the propagation of both finite and infinitesimal disturbances. Two distinct regimes are detected: in the former case, both…
Complex systems are characterized by multiple spatial and temporal scales. A natural framework to capture their multiscale nature is that of multilayer networks, where different layers represent distinct physical processes that often…
Human social behavior is organized in stratified, hierarchical networks, with a support group with about 5 members, expanding proportionally at each layer up to a maximum of approximately 150 frequent interactions per individual. This is…
Many biological phenomena or social events critically depend on how information evolves in complex networks. However, a general theory to characterize information evolution is yet absent. Consequently, numerous unknowns remain about the…
The dynamics of information diffusion within graphs is a critical open issue that heavily influences graph representation learning, especially when considering long-range propagation. This calls for principled approaches that control and…
We investigate the propagation of information through one-dimensional quantum chains in fluctuating external fields. We find that information propagation is suppressed, but in a quite different way compared to the situation with static…
When the parameters are independently and identically distributed (initialized) neural networks exhibit undesirable properties that emerge as the number of layers increases, e.g. a vanishing dependency on the input and a concentration on…
The mean field (MF) theory of multilayer neural networks centers around a particular infinite-width scaling, where the learning dynamics is closely tracked by the MF limit. A random fluctuation around this infinite-width limit is expected…
The study of dynamical systems defined on complex networks provides a natural framework with which to investigate myriad features of neural dynamics, and has been widely undertaken. Typically, however, networks employed in theoretical…