Related papers: Balancing structure and randomness: maximum entrop…

Many socioeconomic phenomena, such as technology adoption, collaborative problem-solving, and content engagement, involve a collection of agents coordinating to take a common action, aligning their decisions to maximize their individual…

The success of deep convolutional neural network (CNN) in computer vision especially image classification problems requests a new information theory for function of image, instead of image itself. In this article, after establishing a deep…

Mechanistic models can provide an intuitive and interpretable explanation of network growth by specifying a set of generative rules. These rules can be defined by domain knowledge about real-world mechanisms governing network growth or may…

Cortical networks exhibit synchronized activity which often occurs in spontaneous events in the form of spike avalanches. Since synchronization has been causally linked to central aspects of brain function such as selective signal…

Exploiting the great expressive power of Deep Neural Network architectures, relies on the ability to train them. While current theoretical work provides, mostly, results showing the hardness of this task, empirical evidence usually differs…

The entropy of the codes usually serves as the rate loss in the recent learned lossy image compression methods. Precise estimation of the probabilistic distribution of the codes plays a vital role in the performance. However, existing deep…

Convolutional neural networks (CNNs) trained with cross-entropy loss have proven to be extremely successful in classifying images. In recent years, much work has been done to also improve the theoretical understanding of neural networks.…

Components connected over a network influence each other and interact in various ways. Examples of such systems are networks of computing nodes, which the nodes interact by exchanging workload, for instance, for load balancing purposes. In…

Training neural networks means solving a high-dimensional optimization problem. Normally the goal is to minimize a loss function that depends on what is called the network function, or in other words the function that gives the network…

Parameter estimation in Markov random fields (MRFs) is a difficult task, in which inference over the network is run in the inner loop of a gradient descent procedure. Replacing exact inference with approximate methods such as loopy belief…

We develop a statistical theory to characterize correlations in weighted networks. We define the appropriate metrics quantifying correlations and show that strictly uncorrelated weighted networks do not exist due to the presence of…

The high computational complexity and increasing parameter counts of deep neural networks pose significant challenges for deployment in resource-constrained environments, such as edge devices or real-time systems. To address this, we…

Our theoretical understanding of neural networks is lagging behind their empirical success. One of the important unexplained phenomena is why and how, during the process of training with gradient descent, the theoretical capacity of neural…

We consider the problem of inferring the functional connectivity of a large-scale computer network from sparse time series of events emitted by its nodes. We do so under the following three domain-specific constraints: (a) non-stationarity…

The connectivity properties of a weight-bearing network are exploited to enhance it's capacity. We study a 2-d network of sites where the weight-bearing capacity of a given site depends on the capacities of the sites connected to it in the…

In systems neuroscience, most models posit that brain regions communicate information under constraints of efficiency. Yet, evidence for efficient communication in structural brain networks characterized by hierarchical organization and…

Recent work on mode connectivity in the loss landscape of deep neural networks has demonstrated that the locus of (sub-)optimal weight vectors lies on continuous paths. In this work, we train a neural network that serves as a hypernetwork,…

In order to assess the short-term memory performance of non-linear random neural networks, we introduce a measure to quantify the dependence of a neural representation upon the past context. We study this measure both numerically and…

The intricate relations between elements in natural and human-made systems sustain the complex processes that shape our world, forming multiscale networks of interactions. These networks can be represented as graphs composed of nodes…

Many complex natural and physical systems exhibit patterns of interconnection that conform, approximately, to a network structure referred to as scale-free. Preferential attachment is one of many algorithms that have been introduced to…