Related papers: Universal Approximation Under Constraints is Possi…
Training deep neural networks for solving machine learning problems is one great challenge in the field, mainly due to its associated optimisation problem being highly non-convex. Recent developments have suggested that many training…
For a compact subset $K$ of the complex plane $\mathbb C,$ let $C(K)$ denote the algebra of continuous functions on $K$. For an open subset $U \subset K,$ let $A(K,U) \subset C(K)$ be the algebra of functions that are analytic in $U.$ We…
We consider applications of neural networks in nonlinear system identification and formulate a hypothesis that adjusting general network structure by incorporating frequency information or other known orthogonal transform, should result in…
We show that a neural network whose output is obtained as the difference of the outputs of two feedforward networks with exponential activation function in the hidden layer and logarithmic activation function in the output node (LSE…
Many problems of theoretical and practical interest involve finding an optimum over a family of convex functions. For instance, finding the projection on the convex functions in $H^k(\Omega)$, and optimizing functionals arising from some…
The problem of maximizing a constrained monotone set function has many practical applications and generalizes many combinatorial problems. Unfortunately, it is generally not possible to maximize a monotone set function up to an acceptable…
Neural networks (NNs) have emerged as powerful tools for solving high-dimensional optimal control problems. In particular, their compositional structure has been shown to enable efficient approximation of high-dimensional functions, helping…
Motivated by the developing mathematics of deep learning, we build universal functions approximators of continuous maps between arbitrary Polish metric spaces $\mathcal{X}$ and $\mathcal{Y}$ using elementary functions between Euclidean…
Randomized Hadamard Transforms (RHTs) have emerged as a computationally efficient alternative to the use of dense unstructured random matrices across a range of domains in computer science and machine learning. For several applications such…
We resolve a long-standing open question, about the existence of a constant-factor approximation algorithm for the average-case \textsc{Decision Tree} problem with uniform probability distribution over the hypotheses. We answer the question…
Standard uniform convergence results bound the generalization gap of the expected loss over a hypothesis class. The emergence of risk-sensitive learning requires generalization guarantees for functionals of the loss distribution beyond the…
We consider the problem of approximating a given element $f$ from a Hilbert space $\mathcal{H}$ by means of greedy algorithms and the application of such procedures to the regression problem in statistical learning theory. We improve on the…
We introduce so-called functional input neural networks defined on a possibly infinite dimensional weighted space with values also in a possibly infinite dimensional output space. To this end, we use an additive family to map the input…
We establish universal approximation theorems for infinite-dimensional geometric rough paths, i.e., we show that continuous functions on the space of infinite-dimensional weakly geometric H\"older continuous rough paths can be approximated…
We analyze the universal approximation constraints of narrow Residual Neural Networks (ResNets) both theoretically and numerically. For deep neural networks without input space augmentation, a central constraint is the inability to…
This article presents a general approximation-theoretic framework to analyze measure transport algorithms for probabilistic modeling. A primary motivating application for such algorithms is sampling -- a central task in statistical…
Motivated by the Maximum Theorem for convex functions (in the setting of linear spaces) and for subadditive functions (in the setting of Abelian semigroups), we establish a Maximum Theorem for the class of generalized convex functions,…
We describe several algorithms for matrix completion and matrix approximation when only some of its entries are known. The approximation constraint can be any whose approximated solution is known for the full matrix. For low rank…
We develop new adaptive algorithms for variational inequalities with monotone operators, which capture many problems of interest, notably convex optimization and convex-concave saddle point problems. Our algorithms automatically adapt to…
We provide conditions ensuring that the KKT-type conditions characterizes the global optimality for quadratically constrained (possibly nonconvex) quadratic programming QCQP problems in Hilbert spaces. The key property is the convexity of a…