Related papers: Critical dynamics governs deep learning

Recent breakthroughs in machine and deep learning (ML and DL) research have provided excellent tools for leveraging enormous amounts of data and optimizing huge models with millions of parameters to obtain accurate networks for image…

Neuromorphic computing seeks to replicate the remarkable efficiency, flexibility, and adaptability of the human brain in artificial systems. Unlike conventional digital approaches, which suffer from the Von Neumann bottleneck and depend on…

The highly non-linear nature of deep neural networks causes them to be susceptible to adversarial examples and have unstable gradients which hinders interpretability. However, existing methods to solve these issues, such as adversarial…

Deep Neural Networks (DNNs) have two key deficiencies, their dependence on high precision computing and their inability to perform sequential learning, that is, when a DNN is trained on a first task and the same DNN is trained on the next…

The hierarchical organization of the brain is a fundamental structural principle, while brain criticality is a leading hypothesis for its collective dynamics. However, the connection between structure and signatures of criticality remains…

Conventional deep learning prioritizes unconstrained optimization, yet biological systems operate under strict metabolic constraints. We propose that these physical constraints shape dynamics to function not as limitations, but as a…

Present day artificial neural architecture search (NAS) strategies are essentially prediction-error-optimized. That holds true for AI functions in general. From the developmental neuroscience perspective, I present evidence for the central…

The remarkable success of Large Language Models (LLMs) in generative tasks has raised fundamental questions about the nature of their acquired capabilities, which often appear to emerge unexpectedly without explicit training. This paper…

Contemporary machine learning models, including large language models, exhibit remarkable capabilities in static tasks yet falter in non-stationary environments due to rigid architectures that hinder continual adaptation and lifelong…

Criticality is hypothesized as a physical mechanism underlying efficient transitions between cortical states and remarkable information processing capacities in the brain. While considerable evidence generally supports this hypothesis,…

This paper presents SYMBIOSIS, an AI-powered framework and platform designed to make Systems Thinking accessible for addressing societal challenges and unlock paths for leveraging systems thinking frameworks to improve AI systems. The…

Complex networks have become essential tools for understanding diverse phenomena in social systems, traffic systems, biomolecular systems, and financial systems. Identifying critical nodes is a central theme in contemporary research,…

The critical brain hypothesis posits that neural systems operate near a phase transition, optimizing the processing of information. While scale invariance and non-Gaussian dynamics--hallmarks of criticality--have been observed in brain…

Much attention has been devoted recently to the generalization puzzle in deep learning: large, deep networks can generalize well, but existing theories bounding generalization error are exceedingly loose, and thus cannot explain this…

Researchers are exploring the integration of IoT and the cloud continuum, together with AI to enhance the cost-effectiveness and efficiency of critical infrastructure (CI) systems. This integration, however, increases susceptibility of CI…

Deep Neural Networks (DNNs) are everywhere, frequently performing a fairly complex task that used to be unimaginable for machines to carry out. In doing so, they do a lot of decision making which, depending on the application, may be…

We introduce a unified theoretical framework for the rigorous analysis and systematic construction of deep neural networks (DNNs). This framework addresses a gap in existing theory by explicitly modeling the structure of tensor operations…

This paper analyzes the robustness of state-of-the-art AI-based models for power grid operations under the $N-1$ security criterion. While these models perform well in regular grid settings, our results highlight a significant loss in…

Empirical estimation of critical points at which complex systems abruptly flip from one state to another is among the remaining challenges in network science. However, due to the stochastic nature of critical transitions it is widely…

The brain forms and stores distributed representations from sparse external input that compete for neuronal resources with already stored memory traces. It is unclear what dynamical properties of neural systems allow formation and…