Related papers: Approximation results on neural network operators …
The choice of activation functions and their motivation is a long-standing issue within the neural network community. Neuronal representations within artificial neural networks are commonly understood as logits, representing the log-odds…
Approximation properties of multivariate Kantorovich-Kotelnikov type operators generated by different band-limited functions are studied. In particular, a wide class of functions with discontinuous Fourier transform is considered. The…
In this work, we introduce a planning neural operator (PNO) for predicting the value function of a motion planning problem. We recast value function approximation as learning a single operator from the cost function space to the value…
In this article, we analyze the approximation properties of the new family of Durrmeyer type exponential sampling operators. We derive the point-wise and uniform approximation theorem and Voronovskaya type theorem for these generalized…
We develop a general, functional calculus approach to approximation of $C_0$-semigroups on Banach spaces by bounded completely monotone functions of their generators. The approach comprises most of well-known approximation formulas, yields…
Recent advances in convolutional neural networks have considered model complexity and hardware efficiency to enable deployment onto embedded systems and mobile devices. For example, it is now well-known that the arithmetic operations of…
We perform conformal perturbation theory by marginal operators to first order. A suitable renormalization method is needed that makes the conformal invariance of the deformed correlation functions manifest. Combining the embedding space…
Coupled oscillators are being increasingly used as the basis of machine learning (ML) architectures, for instance in sequence modeling, graph representation learning and in physical neural networks that are used in analog ML devices. We…
We study the complexity of closure operators, with applications to machine learning and decision theory. In machine learning, closure operators emerge naturally in data classification and clustering. In decision theory, they can model…
Training feedforward neural networks with standard logistic activations is considered difficult because of the intrinsic properties of these sigmoidal functions. This work aims at showing that these networks can be trained to achieve…
Spatial relations between objects in an image have proved useful for structural object recognition. Structural constraints can act as regularization in neural network training, improving generalization capability with small datasets.…
In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closed-form…
A neural network computes a function. A central property of neural networks is that they are "universal approximators:" for a given continuous function, there exists a neural network that can approximate it arbitrarily well, given enough…
Extending the notion of parallelism we introduce the concept of approximate parallelism in normed spaces and then substantially restrict ourselves to the setting of Hilbert space operators endowed with the operator norm. We present several…
Real and complex norms of a linear operator acting on a normed complexified space are considered. Bounds on the ratio of these norms are given. The real and complex norms are shown to coincide for four classes of operators: 1) real linear…
The paper briefy reviews several recent results on hierarchical architectures for learning from examples, that may formally explain the conditions under which Deep Convolutional Neural Networks perform much better in function approximation…
Batch Normalization (BN) improves both convergence and generalization in training neural networks. This work understands these phenomena theoretically. We analyze BN by using a basic block of neural networks, consisting of a kernel layer, a…
Quadratic-support functions [Aravkin, Burke, and Pillonetto; J. Mach. Learn. Res. 14(1), 2013] constitute a parametric family of convex functions that includes a range of useful regularization terms found in applications of convex…
Deep neural networks have emerged as powerful tools for learning operators defined over infinite-dimensional function spaces. However, existing theories frequently encounter difficulties related to dimensionality and limited…
Monotone operators, especially in the form of subdifferential operators, are of basic importance in optimization. It is well known since Minty, Rockafellar, and Bertsekas-Eckstein that in Hilbert space, monotone operators can be understood…