Related papers: A duality connecting neural network and cosmologic…
Dynamical systems methods are used to investigate global behavior of the spatially flat Friedmann-Robertson-Walker cosmological model in gravitational theory with a non-minimally coupled scalar field and a constant potential function. We…
A quantum neural network (QNN) is a parameterized mapping efficiently implementable on near-term Noisy Intermediate-Scale Quantum (NISQ) computers. It can be used for supervised learning when combined with classical gradient-based…
We describe a novel duality symmetry of Phi(4)-theory defined on noncommutative Euclidean space and with noncommuting momentum coordinates. This duality acts on the fields by Fourier transformation and scaling. It is an extension, to…
We study a mathematical model of biological neuronal networks composed by any finite number $N \geq 2$ of non necessarily identical cells. The model is a deterministic dynamical system governed by finite-dimensional impulsive differential…
Machine learning has shown significant breakthroughs in quantum science, where in particular deep neural networks exhibited remarkable power in modeling quantum many-body systems. Here, we explore how the capacity of data-driven deep neural…
In artificial neural networks, the activation dynamics of non-trainable variables is strongly coupled to the learning dynamics of trainable variables. During the activation pass, the boundary neurons (e.g., input neurons) are mapped to the…
This paper explores learned-context neural networks. It is a multi-task learning architecture based on a fully shared neural network and an augmented input vector containing trainable task parameters. The architecture is interesting due to…
Cortical neurons whose activity is recorded in behavioral experiments has been classified into several types such as stimulus-related neurons, delay-period neurons, and reward-related neurons. Moreover, the population activity of neurons…
We propose a hierarchically modular, dynamical neural network model whose architecture minimizes a specifically designed energy function and defines its temporal characteristics. The model has an internal and an external space that are…
This paper investigates the double descent phenomenon in two-layer neural networks, focusing on the role of L1 regularization and representation dimensions. It explores an alternative double descent phenomenon, named sparse double descent.…
Dynamical networks are powerful tools for modeling a broad range of complex systems, including financial markets, brains, and ecosystems. They encode how the basic elements (nodes) of these systems interact altogether (via links) and evolve…
While a lot of progress has been made in recent years, the dynamics of learning in deep nonlinear neural networks remain to this day largely misunderstood. In this work, we study the case of binary classification and prove various…
Recurrent Neural Networks (RNNs) are widely used for modelling neural activity, yet the mathematical interplay of core procedures is used to analyze them (temporal rescaling, discretization, and linearization) remain uncharacterized. This…
Recurrent neural networks (RNNs) trained on neuroscience-inspired tasks offer powerful models of brain computation. However, typical training paradigms rely on open-loop, supervised settings, whereas real-world learning unfolds in…
Neural scaling laws describe how the performance of deep neural networks scales with key factors such as training data size, model complexity, and training time, often following power-law behaviors over multiple orders of magnitude. Despite…
The ability to train ever-larger neural networks brings artificial intelligence to the forefront of scientific and technical discoveries. However, their exponentially increasing size creates a proportionally greater demand for energy and…
We propose an empirical approach centered on the spectral dynamics of weights -- the behavior of singular values and vectors during optimization -- to unify and clarify several phenomena in deep learning. We identify a consistent bias in…
Complex dynamical systems are often modeled as networks, with nodes representing dynamical units which interact through the network's links. Gene regulatory networks, responsible for the production of proteins inside a cell, are an example…
We introduce a dynamical model to reduce a large cosmological constant to a sufficiently small value. The basic ingredient in this model is a distinction which has been made between the two unit systems used in cosmology and particle…
When training deep neural networks with gradient descent, sharpness often increases -- a phenomenon known as progressive sharpening -- before saturating at the edge of stability. Although commonly observed in practice, the underlying…