Related papers: Criticality & Deep Learning I: Generally Weighted …

We investigate the critical properties of kinetic continuous opinion dynamics using deep learning techniques. The system consists of $N$ continuous spin variables in the interval $[-1,1]$. Dense neural networks are trained on spin…

The "edge of chaos" phase transition in artificial neural networks is of renewed interest in light of recent evidence for criticality in brain dynamics. Statistical mechanics traditionally studied this transition with connectivity $k$ as…

Living systems operate in a critical dynamical regime -- between order and chaos -- where they are both resilient to perturbation, and flexible enough to evolve. To characterize such critical dynamics, the established 'structural theory' of…

The combination of the compactness of networks, featuring small diameters, and their complex architectures results in a variety of critical effects dramatically different from those in cooperative systems on lattices. In the last few years,…

Disordered fibrous networks are ubiquitous in nature as major structural components of living cells and tissues. The mechanical stability of networks generally depends on the degree of connectivity: only when the average number of…

We demonstrate, both analytically and numerically, that learning dynamics of neural networks is generically attracted towards a self-organized critical state. The effect can be modeled with quartic interactions between non-trainable…

Continual learning of a stream of tasks is an active area in deep neural networks. The main challenge investigated has been the phenomenon of catastrophic forgetting or interference of newly acquired knowledge with knowledge from previous…

Gene regulatory networks can be successfully modeled as Boolean networks. A much discussed hypothesis says that such model networks reproduce empirical findings the best if they are tuned to operate at criticality, i.e. at the borderline…

Deep learning models have been criticized for their lack of easy interpretation, which undermines confidence in their use for important applications. Nevertheless, they are consistently utilized in many applications, consequential to…

Deep neural networks give us a powerful method to model the training dataset's relationship between input and output. We can regard that as a complex adaptive system consisting of many artificial neurons that work as an adaptive memory as a…

Over the past several years, there have been many studies demonstrating the ability of deep neural networks to identify phase transitions in many physical systems, notably in classical statistical physics systems. One often finds that the…

Sensitivity to small changes in the environment is crucial for many real-world tasks, enabling living and artificial systems to make correct behavioral decisions. It has been shown that such sensitivity is maximized when a system operates…

Interactions among people or objects are often dynamic in nature and can be represented as a sequence of networks, each providing a snapshot of the interactions over a brief period of time. An important task in analyzing such evolving…

Much attention has been devoted recently to the generalization puzzle in deep learning: large, deep networks can generalize well, but existing theories bounding generalization error are exceedingly loose, and thus cannot explain this…

The cooperative behaviour of interacting neurons and synapses is studied using models and methods from statistical physics. The competition between training error and entropy may lead to discontinuous properties of the neural network. This…

It has long been argued that neural networks have to establish and maintain a certain intermediate level of activity in order to keep away from the regimes of chaos and silence. Strong evidence for criticality has been observed in terms of…

With the rapid advancements of deep learning in the past decade, it can be foreseen that deep learning will be continuously deployed in more and more safety-critical applications such as autonomous driving and robotics. In this context,…

Deep learning has sparked a network of mutual interactions between different disciplines and AI. Naturally, each discipline focuses and interprets the workings of deep learning in different ways. This diversity of perspectives on deep…

Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning. In this work, we propose a theoretical framework to analyze learning dynamics through the lens of dynamical…

Recent observation for scale invariant neural avalanches in the brain have been discussed in details in the scientific literature. We point out, that these results do not necessarily imply that the properties of the underlying neural…