Related papers: Nonlinear Approximation via Compositions
Constructing neural networks for function approximation is a classical and longstanding topic in approximation theory. In this paper, we aim at constructing deep neural networks (deep nets for short) with three hidden layers to approximate…
Multi-task learning (MTL) has emerged as a pivotal paradigm in machine learning by leveraging shared structures across multiple related tasks. Despite its empirical success, the development of likelihood-based efficiently solvable…
Symmetric submodular functions are an important family of submodular functions capturing many interesting cases including cut functions of graphs and hypergraphs. Maximization of such functions subject to various constraints receives little…
Restricted non-linear approximation is a type of N-term approximation where a measure $\nu$ on the index set (rather than the counting measure) is used to control the number of terms in the approximation. We show that embeddings for…
We propose Top-N-Rank, a novel family of list-wise Learning-to-Rank models for reliably recommending the N top-ranked items. The proposed models optimize a variant of the widely used discounted cumulative gain (DCG) objective function which…
For the past 30 years or so, machine learning has stimulated a great deal of research in the study of approximation capabilities (expressive power) of a multitude of processes, such as approximation by shallow or deep neural networks,…
Several questions of approximation theory are discussed: 1) can one approximate stably in $L^\infty$ norm $f^\prime$ given approximation $f_\delta, \parallel f_\delta - f \parallel_{L^\infty} < \delta$, of an unknown smooth function $f(x)$,…
This survey provides an in-depth and explanatory review of the approximation properties of deep neural networks, with a focus on feed-forward and residual architectures. The primary objective is to examine how effectively neural networks…
This paper proposes a novel localized Fourier extension method for approximating non-periodic functions via domain segmentation. By partitioning the computational domain into subregions with uniform discretization scales, the method…
A common approach for minimizing a smooth nonlinear function is to employ finite-difference approximations to the gradient. While this can be easily performed when no error is present within the function evaluations, when the function is…
Function fitting/approximation plays a fundamental role in computer graphics and other engineering applications. While recent advances have explored neural networks to address this task, these methods often rely on architectures with many…
Compositionality in language refers to how much the meaning of some phrase can be decomposed into the meaning of its constituents and the way these constituents are combined. Based on the premise that substitution by synonyms is…
Recent progress in deep learning has been driven by increasingly larger models. However, their computational and energy demands have grown proportionally, creating significant barriers to their deployment and to a wider adoption of deep…
A functional differential equation related to the logistic equation is studied by a combination of numerical and perturbation methods. Parameter regions are identified where the solution to the nonlinear problem is approximated well by…
This work proposes a machine-learning framework for constructing statistical models of errors incurred by approximate solutions to parameterized systems of nonlinear equations. These approximate solutions may arise from early termination of…
Recent years have witnessed the introduction and development of extremely fast rational function algorithms. Many ideas in this realm arose from polynomial-based linear-algebraic algorithms. However, polynomial approximation is occasionally…
In this paper, we introduce maximum composition ordering problems. The input is $n$ real functions $f_1,\dots,f_n:\mathbb{R}\to\mathbb{R}$ and a constant $c\in\mathbb{R}$. We consider two settings: total and partial compositions. The…
Training a one-node neural network with ReLU activation function (One-Node-ReLU) is a fundamental optimization problem in deep learning. In this paper, we begin with proving the NP-hardness of training One-Node-ReLU. We then present an…
We present a method for constructing global analytical expressions that approximate a function over its entire range. These approximations not only mirror the original function as accurately as desired, but are purposefully created to…
Designing and implementing systems as an interconnection of smaller subsystems is a common practice for modularity and standardization of components and design algorithms. Although not typically cast in this framework, many of these…