Related papers: Algorithmic construction of the subdifferential fr…
Steepest descent is central in variational mathematics. We present a new transparent existence proof for curves of near-maximal slope --- an influential notion of steepest descent in a nonsmooth setting. We moreover show that for…
We consider the regression problem of estimating functions on $\mathbb{R}^D$ but supported on a $d$-dimensional manifold $ \mathcal{M} \subset \mathbb{R}^D $ with $ d \ll D $. Drawing ideas from multi-resolution analysis and nonlinear…
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of…
Variational inequalities play a key role in machine learning research, such as generative adversarial networks, reinforcement learning, adversarial training, and generative models. This paper is devoted to the constrained variational…
The subgradient method is one of the most fundamental algorithmic schemes for nonsmooth optimization. The existing complexity and convergence results for this method are mainly derived for Lipschitz continuous objective functions. In this…
Evaluating a polynomial on a set of points is a fundamental task in computer algebra. In this work, we revisit a particular variant called trimmed multipoint evaluation: given an $n$-variate polynomial with bounded individual degree $d$ and…
The paper is devoted to an analysis of optimality conditions for nonsmooth multidimensional problems of the calculus of variations with various types of constraints, such as additional constraints at the boundary and isoperimetric…
Improvements in technology lead to increasing availability of large data sets which makes the need for data reduction and informative subsamples ever more important. In this paper we construct $ D $-optimal subsampling designs for…
In this paper, a globally convergent trust region proximal gradient method is developed for composite multi-objective optimization problems where each objective function can be represented as the sum of a smooth function and a nonsmooth…
We give upper bounds on the Walsh coefficients of functions for which the derivative of order at least one has bounded variation of fractional order. Further, we also consider the Walsh coefficients of functions in periodic and non-periodic…
Differentiation is a cornerstone of computing and data analysis in every discipline of science and engineering. Indeed, most fundamental physics laws are expressed as relationships between derivatives in space and time. However, derivatives…
Fractional derivatives are a well-studied generalization of integer order derivatives. Naturally, for optimization, it is of interest to understand the convergence properties of gradient descent using fractional derivatives. Convergence…
In this paper we generalize the involutive methods and algorithms devised for polynomial ideals to differential ones generated by a finite set of linear differential polynomials in the differential polynomial ring over a zero characteristic…
In a nutshell, submodular functions encode an intuitive notion of diminishing returns. As a result, submodularity appears in many important machine learning tasks such as feature selection and data summarization. Although there has been a…
Prediction-correction algorithms are a highly effective class of methods for solving pseudo-convex optimization problems. The descent direction of these algorithms can be viewed as an adjustment to the gradient direction based on the…
A new algorithm for one-dimensional minimization is described in detail and the results of some tests on practical cases are reported and illustrated. The method requires only punctual computation of the function, and is suitable to be…
The known Complex Step Derivative (CSD) method allows easy and accurate differentiation up to machine precision of real analytic functions by evaluating them a small imaginary step next to the real number line. The current paper proposes…
We describe and study geometric properties of discrete circular and spherical means of directional derivatives of functions, as well as discrete approximations of higher order differential operators. For an arbitrary dimension we present a…
This paper concerns dictionary learning, i.e., sparse coding, a fundamental representation learning problem. We show that a subgradient descent algorithm, with random initialization, can provably recover orthogonal dictionaries on a natural…
A new representation is proposed for functions in a Sobolev space with dominating mixed smoothness on an $N$-dimensional hyperrectangle. In particular, it is shown that these functions can be expressed in terms of their highest-order mixed…