Related papers: Criticality & Deep Learning I: Generally Weighted …

Critical learning periods are periods early in development where temporary sensory deficits can have a permanent effect on behavior and learned representations. Despite the radical differences between biological and artificial networks,…

Similar to humans and animals, deep artificial neural networks exhibit critical periods during which a temporary stimulus deficit can impair the development of a skill. The extent of the impairment depends on the onset and length of the…

The rapid advances in artificial intelligence (AI) have largely been driven by scaling deep neural networks (DNNs) - increasing model size, data, and computational resources. Yet performance is ultimately governed by network dynamics. The…

This study delves into the plasticity of neural networks, offering empirical support for the notion that critical learning periods and warm-starting performance loss can be avoided through simple adjustments to learning hyperparameters. The…

Systems with different interactions could develop the same critical behaviour due to the underlying symmetry and universality. Using this principle of universality, we can embed critical correlations modeled on the 3D Ising model into the…

A universal supervised neural network (NN) relevant to compute the associated criticalities of real experiments studying phase transitions is constructed. The validity of the built NN is examined by applying it to calculate the…

Critical Learning Periods comprehend an important phenomenon involving deep learning, where early epochs play a decisive role in the success of many training recipes, such as data augmentation. Existing works confirm the existence of this…

In recent years, machine learning has been adopted to complex networks, but most existing works concern about the structural properties. To use machine learning to detect phase transitions and accurately identify the critical transition…

The observation of critical-like behavior in cortical networks represents a major step forward in elucidating how the brain manages information. Understanding the origin and functionality of critical-like dynamics, as well as their…

As a promising computational paradigm, occurrence of critical states in artificial and biological neural networks has attracted wide-spread attention. An often-made explicit or implicit assumption is that one single critical state is…

We show that the ability of a neural network to integrate information from diverse sources hinges critically on being exposed to properly correlated signals during the early phases of training. Interfering with the learning process during…

Despite their prevalence, deep networks are poorly understood. This is due, at least in part, to their highly parameterized nature. As such, while certain structures have been found to work better than others, the significance of a model's…

Deep neural networks are strongly over-parameterized, often containing far more weights than required for their task. Although such redundancy can aid optimization, it leads to inefficient deployment and high computational cost, motivating…

Centrality is an important notion in network analysis and is used to measure the degree to which network structure contributes to the importance of a node in a network. While many different centrality measures exist, most of them apply to…

Criticality can be exactly demonstrated in certain models of brain activity, yet it remains challenging to identify in empirical data. We trained a fully connected deep neural network to learn the phases of an excitable model unfolding on…

Empirical estimation of critical points at which complex systems abruptly flip from one state to another is among the remaining challenges in network science. However, due to the stochastic nature of critical transitions it is widely…

Hallmarks of criticality, such as power-laws and scale invariance, have been empirically found in cortical networks and it has been conjectured that operating at criticality entails functional advantages, such as optimal computational…

Novel computing hardwares are necessary to keep up with today's increasing demand for data storage and processing power. In this research project, we turn to the brain for inspiration to develop novel computing substrates that are…

Deep learning has arguably achieved tremendous success in recent years. In simple words, deep learning uses the composition of many nonlinear functions to model the complex dependency between input features and labels. While neural networks…

Dynamical phase transitions (DPTs) characterize critical changes in system behavior occurring at finite times, providing a lens to study nonequilibrium phenomena beyond conventional equilibrium physics. While extensively studied in quantum…