Related papers: Physical networks become what they learn
Deep neural networks (DNNs) have achieved exceptional performance across various fields by learning complex, nonlinear mappings from large-scale datasets. However, they face challenges such as high memory requirements and computational…
Network routing approaches are widely used to study the evolution in time of self-adapting systems. However, few advances have been made for problems where adaptation is governed by time-dependent inputs. In this work we study a dynamical…
We study a material modeled as a network of nodes connected by edges. Using a discrete approach, we build a nonlinear algebraic system that connects applied forces to internal forces and node positions. The model can describe elasticity,…
Efficient networking has a substantial economic and societal impact in a broad range of areas including transportation systems, wired and wireless communications and a range of Internet applications. As transportation and communication…
Physics-informed neural networks approach the approximation of differential equations by directly incorporating their structure and given conditions in a loss function. This enables conditions like, e.g., invariants to be easily added…
Given an existing trained neural network, it is often desirable to learn new capabilities without hindering performance of those already learned. Existing approaches either learn sub-optimal solutions, require joint training, or incur a…
The computational modeling of high-speed flows (e.g. hypersonic) and space plasmas is characterized by a plethora of complex physical phenomena, in particular involving strong oblique shocks, bow shocks and/or shock waves boundary layer…
We study adaptive network models in which coupling weights evolve on a fast time scale relative to the phase dynamics of the nodes. Using Geometric Singular Perturbation Theory (GSPT), we prove that, although the microscopic system is…
We present E NERGY N ET , a new framework for analyzing and building artificial neural network architectures. Our approach adaptively learns the structure of the networks in an unsupervised manner. The methodology is based upon the…
To make sense of the world our brains must analyze high-dimensional datasets streamed by our sensory organs. Because such analysis begins with dimensionality reduction, modelling early sensory processing requires biologically plausible…
Deep neural networks (DNN) have been used to model nonlinear relations between physical quantities. Those DNNs are embedded in physical systems described by partial differential equations (PDE) and trained by minimizing a loss function that…
We study the effect of mini-batching on the loss landscape of deep neural networks using spiked, field-dependent random matrix theory. We demonstrate that the magnitude of the extremal values of the batch Hessian are larger than those of…
Adaptive networks are a versatile approach to model phenomena such as contagion and spreading dynamics, critical transitions and structure formation that emerge from the dynamic coevolution of complex network structure and node states.…
Even though neural networks enjoy widespread use, they still struggle to learn the basic laws of physics. How might we endow them with better inductive biases? In this paper, we draw inspiration from Hamiltonian mechanics to train models…
Deep learning approaches, known for their ability to model complex relationships and fast execution, are increasingly being applied to solve large optimization problems. However, existing methods often face challenges in simultaneously…
All living things exhibit adaptations that enable them to survive and reproduce in the natural environment that they inhabit. From a biological standpoint, it has long been understood that adaptation comes from natural selection, whereby…
The Hessian of a neural network captures parameter interactions through second-order derivatives of the loss. It is a fundamental object of study, closely tied to various problems in deep learning, including model design, optimization, and…
Mathematical modeling has broad applications in neuroscience whether modeling the dynamics of a single synapse or an entire network of neurons. In Part I, we model vesicle replenishment and release at the photoreceptor synapse to better…
Neural models learn representations of high-dimensional data on low-dimensional manifolds. Multiple factors, including stochasticities in the training process, model architectures, and additional inductive biases, may induce different…
In this paper, we propose an optimization framework to design a network of positive linear systems whose structure switches according to a Markov process. The optimization framework herein proposed allows the network designer to optimize…