Related papers: A Spin Glass Characterization of Neural Networks
Spontaneous symmetry breaking in statistical mechanics primarily occurs during phase transitions at the thermodynamic limit where the Hamiltonian preserves inversion symmetry, yet the low-temperature free energy exhibits reduced symmetry.…
Statistical mechanics has made significant contributions to the study of biological neural systems by modeling them as recurrent networks of interconnected units with adjustable interactions. Several algorithms have been proposed to…
Glasses offer a broad range of tunable thermophysical properties that are linked to their compositions. However, it is challenging to establish a universal composition-property relation of glasses due to their enormous composition and…
Recent genomic and bioinformatic advances have motivated the development of numerous random network models purporting to describe graphs of biological, technological, and sociological origin. The success of a model has been evaluated by how…
As a surrogate for computationally intensive meso-scale simulation of woven composites, this article presents Recurrent Neural Network (RNN) models. Leveraging the power of transfer learning, the initialization challenges and sparse data…
The mean field theory of a spin glass with a specific form of nearest and next nearest neighbor interactions is investigated. Depending on the sign of the interaction matrix chosen we either find the continuous replica symmetry breaking…
We argue that many properties of fully-connected feedforward neural networks (FCNNs), also called multi-layer perceptrons (MLPs), are explainable from the analysis of a single pair of operations, namely a random projection into a…
One of the most prominent attributes of Neural Networks (NNs) constitutes their capability of learning to extract robust and descriptive features from high dimensional data, like images. Hence, such an ability renders their exploitation as…
Across many scientific and engineering disciplines, it is important to consider how much the output of a given system changes due to perturbations of the input. Here, we investigate the glassy phase of $\pm J$ spin glasses at zero…
The difficult problem of relating the static structure of glassy liquids and their dynamics is a good target for Machine Learning, an approach which excels at finding complex patterns hidden in data. Indeed, this approach is currently a hot…
Recurrent neural networks (RNNs) are brain-inspired models widely used in machine learning for analyzing sequential data. The present work is a contribution towards a deeper understanding of how RNNs process input signals using the response…
This paper describes how realistic neuromorphic networks can have their connectivity fully characterized in analytical fashion. By assuming that all neurons have the same shape and are regularly distributed along the two-dimensional…
In this work we study spin-glass (SG) like behavior in the dynamics of multiple agents in a social or economic context using interactions which are similar to the physical case. The different preferences shown by individual agents are…
The Synthetic Nervous System (SNS) is a biologically inspired neural network (NN). Due to its capability of capturing complex mechanisms underlying neural computation, an SNS model is a candidate for building compact and interpretable NN…
Emergence of deep neural networks (DNNs) has raised enormous attention towards artificial neural networks (ANNs) once again. They have become the state-of-the-art models and have won different machine learning challenges. Although these…
Neural networks that can capture key principles underlying brain computation offer exciting new opportunities for developing artificial intelligence and brain-like computing algorithms. Such networks remain biologically plausible while…
Spin glass (SG) is a typical magnetic system with frozen random spin orientation at low temperatures. The system exhibits rich physical properties, such as infinite number of ground states, memory effect and aging phenomena. There are two…
Randomness and frustration are considered to be the key ingredients for the existence of spin glass (SG) phase. In a canonical system, these ingredients are realized by the random mixture of ferromagnetic (FM) and antiferromagnetic (AF)…
Hypercomplex neural networks are gaining increasing interest in the deep learning community. The attention directed towards hypercomplex models originates from several aspects, spanning from purely theoretical and mathematical…
We analyze the performance of graph neural network (GNN) architectures from the perspective of random graph theory. Our approach promises to complement existing lenses on GNN analysis, such as combinatorial expressive power and worst-case…