Related papers: Ising models of deep neural networks

Thermodynamics-informed neural networks employ inductive biases for the enforcement of the first and second principles of thermodynamics. To construct these biases, a metriplectic evolution of the system is assumed. This provides excellent…

In order to investigate the role of the weight in weighted networks, the collective behavior of the Ising system on weighted regular networks is studied by numerical simulation. In our model, the coupling strength between spins is inversely…

The zero temperature quenching dynamics of the ferromagnetic Ising model on a densely connected small world network is studied where long range bonds are added randomly with a finite probability $p$. We find that in contrast to the sparsely…

Neural networks based on metric recognition methods have a strictly determined architecture. Number of neurons, connections, as well as weights and thresholds values are calculated analytically, based on the initial conditions of tasks:…

Recent studies have shown that high disparities in effective learning rates (ELRs) across layers in deep neural networks can negatively affect trainability. We formalize how these disparities evolve over time by modeling weight dynamics…

We investigate the asymptotic properties of deep Residual networks (ResNets) as the number of layers increases. We first show the existence of scaling regimes for trained weights markedly different from those implicitly assumed in the…

Scale-free and non-computable characteristics of natural networks are found to result from the least-time dispersal of energy. To consider a network as a thermodynamic system is motivated since ultimately everything that exists can be…

Brain "rest" is defined -more or less unsuccessfully- as the state in which there is no explicit brain input or output. This work focuss on the question of whether such state can be comparable to any known \emph{dynamical} state. For that…

Network reliability is the probability that a dynamical system composed of discrete elements interacting on a network will be found in a configuration that satisfies a particular property. We introduce a new reliability property, Ising…

Deep neural networks come in many sizes and architectures. The choice of architecture, in conjunction with the dataset and learning algorithm, is commonly understood to affect the learned neural representations. Yet, recent results have…

Machine learning is advancing towards a data-science approach, implying a necessity to a line of investigation to divulge the knowledge learnt by deep neuronal networks. Limiting the comparison among networks merely to a predefined…

Neural Networks (NN) has been used in many areas with great success. When a NN's structure (Model) is given, during the training steps, the parameters of the model are determined using an appropriate criterion and an optimization algorithm…

We explore a one-to-one correspondence between a neural network (NN) and a statistical mechanical spin model where neurons are mapped to Ising spins and weights to spin-spin couplings. The process of training an NN produces a family of spin…

Hypernetworks, or hypernets for short, are neural networks that generate weights for another neural network, known as the target network. They have emerged as a powerful deep learning technique that allows for greater flexibility,…

In this paper, we consider the problem of exploring structural regularities of networks by dividing the nodes of a network into groups such that the members of each group have similar patterns of connections to other groups. Specifically,…

Machine learning has become a central technique for modeling in science and engineering, either complementing or as surrogates to physics-based models. Significant efforts have recently been devoted to models capable of predicting field…

An InfoCGAN neural network is trained on 2-dimensional square Ising configurations conditioned on the external applied magnetic field and the temperature. The network is composed of two main sub-networks. The generator network learns to…

Virtually every organism gathers information about its noisy environment and builds models from that data, mostly using neural networks. Here, we use stochastic thermodynamics to analyse the learning of a classification rule by a neural…

We introduce a framework for reasoning about what meaning is captured by the neurons in a trained neural network. We provide a strategy for discovering meaning by training a second model (referred to as an observer model) to classify the…

Machine Learning methods and, in particular, Artificial Neural Networks (ANNs) have demonstrated promising capabilities in material constitutive modeling. One of the main drawbacks of such approaches is the lack of a rigorous frame based on…