相关论文: Comparing Information-Theoretic Measures of Comple…

We consider a generalization of the Hopfield model, where the entries of patterns are Gaussian and diluted. We focus on the high-storage regime and we investigate analytically the topological properties of the emergent network, as well as…

In \cite{Hop82}, Hopfield introduced a \emph{Hebbian} learning rule based neural network model and suggested how it can efficiently operate as an associative memory. Studying random binary patterns, he also uncovered that, if a small…

In biological systems, neuromodulation tunes synaptic plasticity based on the internal state of the organism, complementing stimulus-driven Hebbian learning. The algorithm recently proposed by Krotov and Hopfield \cite{krotov_2019} can be…

Transfer learning methods start performing poorly when the complexity of the learning task is increased. Most of these methods calculate the cumulative differences of all the matched features and then use them to back-propagate that loss…

We consider the training process of a neural network as a dynamical system acting on the high-dimensional weight space. Each epoch is an application of the map induced by the optimization algorithm and the loss function. Using this induced…

The purpose of this study is to investigate how homophily, memory constraints, and adversarial disruptions collectively shape the resilience and adaptability of complex networks. To achieve this, we develop a new framework that integrates…

It has been demonstrated that one of the most striking features of the nervous system, the so called 'plasticity' (i.e high adaptability at different structural levels) is primarily based on Hebbian learning which is a collection of…

Comparing networks is essential for a number of downstream tasks, from clustering to anomaly detection. Despite higher-order interactions being critical for understanding the dynamics of complex systems, traditional approaches for network…

Recently it has been argued that entropy can be a direct measure of complexity, where the smaller value of entropy indicates lower system complexity, while its larger value indicates higher system complexity. We dispute this view and…

We define the complexity of a continuous-time linear system to be the minimum number of bits required to describe its forward increments to a desired level of fidelity, and compute this quantity using the rate distortion function of a…

Analytical approaches to model the structure of complex networks can be distinguished into two groups according to whether they consider an intensive (e.g., fixed degree sequence and random otherwise) or an extensive (e.g., adjacency…

Working with causal models at different levels of abstraction is an important feature of science. Existing work has already considered the problem of expressing formally the relation of abstraction between causal models. In this paper, we…

Learning new information without forgetting prior knowledge is central to human intelligence. In contrast, neural network models suffer from catastrophic forgetting: a significant degradation in performance on previously learned tasks when…

Hebbian learning is a biological principle that intuitively describes how neurons adapt their connections through repeated stimuli. However, when applied to machine learning, it suffers serious issues due to the unconstrained updates of the…

One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of…

Hebbian and anti-Hebbian plasticity are widely observed in the biological brain, yet their theoretical understanding remains limited. In this work, we find that when a learning method is regularized with L2 weight decay, its learning signal…

Deep Boltzmann machines (DBMs), one of the first ``deep'' learning methods ever studied, are multi-layered probabilistic models governed by a pairwise energy function that describes the likelihood of all variables/nodes in the network. In…

This Thesis explores how tools from Statistical Physics and Information Theory can help us describe and understand complex systems. In the first part, we study the interplay between internal interactions, environmental changes, and…

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network…

Understanding the complex hierarchical topology of functional brain networks is a key aspect of functional connectivity research. Such topics are obscured by the widespread use of sparse binary network models which are fundamentally…