Related papers: Activation Saturation and Floquet Spectrum Collaps…
Existing works on the expressive power of neural networks typically assume real parameters and exact operations. In this work, we study the expressive power of quantized networks under discrete fixed-point parameters and inexact fixed-point…
We propose a novel methodology to solve a key eigenvalue optimization problem which arises in the contractivity analysis of neural ODEs. When looking at contractivity properties of a one layer weight-tied neural ODE…
We investigate the effects of the saturation boundary on small-x evolution at the next-to-leading order accuracy and beyond. We demonstrate that the instabilities of the next-to-leading order BFKL evolution are not cured by the presence of…
This paper studies the approximation capacity of neural networks with an arbitrary activation function and with norm constraint on the weights. Upper and lower bounds on the approximation error of these networks are computed for smooth…
Pre-trained Transformers often exhibit over-confidence in source patterns and difficulty in forming new target-domain patterns during fine-tuning. We formalize the mechanism of output saturation leading to gradient suppression through…
We analyze the linear response of a half filled Landau level to long wavelength and low frequency driving forces, using Fermi liquid theory for composite fermions. This response is determined by the composite fermions quasi--particle…
There has been a growing interest in expressivity of deep neural networks. However, most of the existing work about this topic focuses only on the specific activation function such as ReLU or sigmoid. In this paper, we investigate the…
We study numerically the effects of time delay in networks of delay-coupled excitable FitzHugh Nagumo systems with dissipation. The generation of periodic self-sustained oscillations and its threshold are analyzed depending on the…
The continuous integration of experimental data into coherent models of the brain is an increasing challenge of modern neuroscience. Such models provide a bridge between structure and activity, and identify the mechanisms giving rise to…
A new non-linear variant of a quantitative extension of the uniform boundedness principle is used to show sharpness of error bounds for univariate approximation by sums of sigmoid and ReLU functions. Single hidden layer feedforward neural…
We study the strong disorder regime of Floquet topological systems in dimension two, that describe independent electrons on a lattice subject to a periodic driving. In the spectrum of the Floquet propagator we assume the existence of an…
We prove a Quantitative Functional Central Limit Theorem for one-hidden-layer neural networks with generic activation function. The rates of convergence that we establish depend heavily on the smoothness of the activation function, and they…
The success of deep networks has been attributed in part to their expressivity: per parameter, deep networks can approximate a richer class of functions than shallow networks. In ReLU networks, the number of activation patterns is one…
Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural…
Networks of model neurons with balanced recurrent excitation and inhibition produce irregular and asynchronous spiking activity. We extend the analysis of balanced networks to include the known dependence of connection probability on the…
In this paper, we introduce a novel non-linear activation function that spontaneously induces class-compactness and regularization in the embedding space of neural networks. The function is dubbed DOME for Difference Of Mirrored Exponential…
We theoretically study the excitation spectrum of confined macroscopic optical lattices in the Mott-insulating limit. For large systems, a fast numerical method is proposed to calculate the ground state filling and excitation energies. We…
We revisit Heisenberg's model for nucleon-nucleon scattering which admits a saturation of the Froissart bound. We examine its uniqueness, and find that up to certain natural generalizations, it is the only action that saturates the bound.…
Objective. Long-term potentiation (LTP) is a fundamental mechanism underlying learning and memory, yet its investigation at the network level in vitro remains challenging, particularly when optogenetic stimulation is used. The objective of…
Efficient pattern separation in dentate gyrus plays an important role in storing information in the hippocampus. Current knowledge of the structure and function of the hippocampus, entorhinal cortex and dentate gyrus, in pattern separation…