Related papers: Parseval Proximal Neural Networks

Proximal operators are of particular interest in optimization problems dealing with non-smooth objectives because in many practical cases they lead to optimization algorithms whose updates can be computed in closed form or very efficiently.…

Information divergences allow one to assess how close two distributions are from each other. Among the large panel of available measures, a special attention has been paid to convex $\varphi$-divergences, such as Kullback-Leibler,…

The paper proposes and justifies a new algorithm of the proximal Newton type to solve a broad class of nonsmooth composite convex optimization problems without strong convexity assumptions. Based on advanced notions and techniques of…

A class of algorithms comprised by certain semismooth Newton and active-set methods is able to solve convex minimization problems involving sparsity-inducing regularizers very rapidly; the speed advantage of methods from this class is a…

This work studies the problem of sparse signal recovery with automatic grouping of variables. To this end, we investigate sorted nonsmooth penalties as a regularization approach for generalized linear models. We focus on a family of sorted…

We study the approximation of functions by tensor networks (TNs). We show that Lebesgue $L^p$-spaces in one dimension can be identified with tensor product spaces of arbitrary order through tensorization. We use this tensor product…

We derive fundamental lower bounds on the connectivity and the memory requirements of deep neural networks guaranteeing uniform approximation rates for arbitrary function classes in $L^2(\mathbb R^d)$. In other words, we establish a…

Universal approximation theorems provide a mathematical explanation for the expressive power of neural networks. They assert that, under mild conditions on the activation function, feedforward neural networks are dense in broad function…

Averaged operators are important in Convex Analysis and Optimization Algorithms. In this paper, we propose classifications of averaged operators, firmly nonexpansive operators, and proximal operators using the Bauschke-Bendit-Moursi modulus…

Using the theory of Hilbert direct integrals, we introduce and study a monotonicity-preserving operation, termed the integral resolvent mixture. It combines arbitrary families of monotone operators acting on different spaces and linear…

This study investigates leveraging stochastic gradient descent (SGD) to learn operators between general Hilbert spaces. We propose weak and strong regularity conditions for the target operator to depict its intrinsic structure and…

\citet{farrell2021deep} establish non-asymptotic high-probability bounds for general deep feedforward neural network (with rectified linear unit activation function) estimators, with \citet[Theorem 1]{farrell2021deep} achieving a suboptimal…

Recovering images from undersampled linear measurements typically leads to an ill-posed linear inverse problem, that asks for proper statistical priors. Building effective priors is however challenged by the low train and test overhead…

Deep Operator Networks (DeepONets) provide a branch-trunk neural architecture for approximating nonlinear operators acting between function spaces. In the classical operator approximation framework, the input is a function $u\in C(K_1)$…

This paper offers a newly created integral approach for operators employing the orthogonal modified Laguerre polynomials and P\u{a}lt\u{a}nea basis. These operators approximate the functions over the interval $[0,\infty)$. Further, the…

Large sectors of the recent optimization literature focused in the last decade on the development of optimal stochastic first order schemes for constrained convex models under progressively relaxed assumptions. Stochastic proximal point is…

The proximal algorithm is a powerful tool to minimize nonlinear and nonsmooth functionals in a general metric space. Motivated by the recent progress in studying the training dynamics of the noisy gradient descent algorithm on two-layer…

The main result of the present theoretical paper is an original decomposition formula for the proximal operator of the sum of two proper, lower semicontinuous and convex functions $f$ and $g$. For this purpose, we introduce a new operator,…

We present an approach to accelerating a wide variety of image processing operators. Our approach uses a fully-convolutional network that is trained on input-output pairs that demonstrate the operator's action. After training, the original…

Since introduced by Martinet and Rockafellar, the proximal point algorithm was generalized in many fruitful directions. More recently, in 2002, Pennanen studied the proximal point algorithm without monotonicity. A year later, Iusem and…