Related papers: Activation Saturation and Floquet Spectrum Collaps…

`Biologically inspired' activation functions, such as the logistic sigmoid, have been instrumental in the historical advancement of machine learning. However in the field of deep learning, they have been largely displaced by rectified…

We study the geometric properties of random neural networks by investigating the boundary volumes of their excursion sets for different activation functions, as the depth increases. More specifically, we show that, for activations which are…

This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning. The ReLU network can…

This paper studies the phenomenon of conduction block in model neurons using high-frequency biphasic stimulation (HFBS). The focus is investigating the triggering of undesired onset action potentials when the HFBS is turned on. The approach…

We build the wrapped Fukaya category W(E) for any monotone symplectic manifold, convex at infinity. We define the open-closed and closed open-string maps. We study their algebraic properties and prove that the string maps are compatible…

We discuss scattering in a CFT via the conformal partial-wave analysis and the Regge limit. The focus of this paper is on understanding an OPE with Minkowski conformal blocks. Starting with a t-channel OPE, it leads to an expansion for an…

Synchronization is an important collective phenomenon in interacting oscillatory agents. Many functional features of the brain are related to synchronization of neurons. The type of synchronization transition that may occur (explosive vs.…

We establish universality and expression rate bounds for a class of neural Deep Operator Networks (DON) emulating Lipschitz (or H\"older) continuous maps $\mathcal G:\mathcal X\to\mathcal Y$ between (subsets of) separable Hilbert spaces…

Optical neural networks usually execute the linear multiply-accumulate operation in the optical domain, whereas the nonlinear activation function is mostly implemented in the digital or electrical domain. Here we demonstrate a broadband…

We analyze a simple one-hidden-layer neural network with ReLU activation functions and fixed biases, with one-dimensional input and output. We study both continuous and discrete versions of the model, and we rigorously prove the convergence…

Previous work has questioned the conditions under which the decision regions of a neural network are connected and further showed the implications of the corresponding theory to the problem of adversarial manipulation of classifiers. It has…

Frozen Density Embedding Theory (FDET) [Wesolowski {\it Phys. Rev. A} {\bf 77}, 012504 (2008)] provides the interpretation of the eigenvalue equations for an embedded $N'$-electron wavefunction, in which the embedding operator is…

Due to presence of magnetic field gradient needed for coherent spin control, dephasing of single-electron spin qubits in silicon quantum dots is often dominated by $1/f$ charge noise. We investigate theoretically fluctuations of ground…

We study the dynamics of excitable integrate-and-fire neurons in a small-world network. At low densities $p$ of directed random connections, a localized transient stimulus results in either self-sustained persistent activity or in a brief…

The Floquet-Bloch theorem allows waves in infinite, lossless periodic media to be expressed as a sum of discrete Floquet-Bloch modes, but its validity is challenged under the realistic constraints of loss and finite extent. In this work, we…

At chemical synapses, an arriving electric signal induces the fusion of vesicles with the presynaptic membrane, thereby releasing neurotransmitters into the synaptic cleft. After a fusion event, both the release site and the vesicle undergo…

Several current bounds on the maximal number of affine regions of a ReLU feed-forward neural network are special cases of the framework [1] which relies on layer-wise activation histogram bounds. We analyze and partially solve a problem in…

This paper establishes a comprehensive approximation result for deep fully-connected neural networks with commonly-used and general activation functions in Sobolev spaces $W^{n,\infty}$, with errors measured in the $W^{m,p}$-norm for $m <…

The un-rectifying technique expresses a non-linear point-wise activation function as a data-dependent variable, which means that the activation variable along with its input and output can all be employed in optimization. The ReLU network…

Closed-loop neural interfaces with on-chip machine learning can detect and suppress disease symptoms in neurological disorders or restore lost functions in paralyzed patients. While high-density neural recording can provide rich neural…