Related papers: A Duality Approach to Regularized Learning Problem…
Bilevel optimization (BLO) problem, where two optimization problems (referred to as upper- and lower-level problems) are coupled hierarchically, has wide applications in areas such as machine learning and operations research. Recently, many…
In this article we investigate the connection between regularization theory for inverse problems and dynamic programming theory. This is done by developing two new regularization methods, based on dynamic programming techniques. The aim of…
We present here a new method for approximating functions defined on superreflexive Banach spaces by differentiable functions with $\alpha$-H\"older derivatives (for some $0<\alpha\leq 1$). The smooth approximation is given by means of an…
An old idea in optimization theory says that since the gradient is a dual vector it may not be subtracted from the weights without first being mapped to the primal space where the weights reside. We take this idea seriously in this paper…
In this paper, we consider the robust linear infinite programming problem $({\rm RLIP}_c) $ defined by \begin{eqnarray*} ({\rm RLIP}_c)\quad &&\inf\; \langle c,x\rangle \textrm{subject to } &&x\in X,\; \langle x^\ast,x \rangle \le r…
We study the non-smooth optimization problems in machine learning, where both the loss function and the regularizer are non-smooth functions. Previous studies on efficient empirical loss minimization assume either a smooth loss function or…
Adaptive cubic regularization methods have emerged as a credible alternative to linesearch and trust-region for smooth nonconvex optimization, with optimal complexity amongst second-order methods. Here we consider a general/new class of…
In this paper we consider the iteratively regularized Gauss-Newton method, where regularization is achieved by Ivanov regularization, i.e., by imposing a priori constraints on the solution. We propose an a posteriori choice of the…
Normalization techniques such as Batch Normalization have been applied successfully for training deep neural networks. Yet, despite its apparent empirical benefits, the reasons behind the success of Batch Normalization are mostly…
We deal with the problem of numerically computing the dual norm, which is important to study sparsity-inducing regularizations (Jenatton et al. 2011,Bach et al. 2012). The dual norms find application in optimization and statistical…
In this work, we consider a class of linear ill-posed problems with operators that map from the sequence space $ \ell_r $ ($r \ge 1$) into a Banach space and in addition satisfy a conditional stability estimate in the scale of sequence…
Disjointly constrained multilinear programming concerns the problem of maximizing a multilinear function on the product of finitely many disjoint polyhedra. While maximizing a linear function on a polytope (linear programming) is known to…
We study a local version of the ball-covering problem in Banach spaces, and obtain a complete solution to it in terms of the norm derivatives. We illustrate the advantage of the local approach by obtaining substantial refinements of several…
In convex optimization, duality theory can sometimes lead to simpler solution methods than those resulting from direct primal analysis. In this paper, this principle is applied to a class of composite variational problems arising in…
We consider the task of computing an approximate minimizer of the sum of a smooth and non-smooth convex functional, respectively, in Banach space. Motivated by the classical forward-backward splitting method for the subgradients in Hilbert…
Convex algebraic geometry concerns the interplay between optimization theory and real algebraic geometry. Its objects of study include convex semialgebraic sets that arise in semidefinite programming and from sums of squares. This article…
We discuss a systematic way to dimensionally regularize divergent sums arising in field theories with an arbitrary number of physical compact dimensions or finite temperature. The method preserves the same symmetries of the action as the…
We propose and analyze a perturbative regularization method to approximate quadratic optimization problems with finite-dimensional degeneracy. The original problem is first approximated by a regularized problem depending on a small positive…
Recently, deep unfolding methods that guide the design of deep neural networks (DNNs) through iterative algorithms have received increasing attention in the field of inverse problems. Unlike general end-to-end DNNs, unfolding methods have…
We prove weak duality between two recent convex relaxation methods for bounding the optimal value of a constrained variational problem in which the objective is an integral functional. The first approach, proposed by Valmorbida et al. (IEEE…