Related papers: More Virtuous Smoothing
In this paper, we consider the general non-oblivious stochastic optimization where the underlying stochasticity may change during the optimization procedure and depends on the point at which the function is evaluated. We develop Stochastic…
Suppose that $f$ belongs to a suitably defined complete metric space $ {{\cal C}}^{{\alpha}}$ of H\"older $ {\alpha}$-functions defined on $[0,1]$. We are interested in whether one can find large (in the sense of Hausdorff, or lower/upper…
In order to solve the minimization of a nonsmooth convex function, we design an inertial second-order dynamic algorithm, which is obtained by approximating the nonsmooth function by a class of smooth functions. By studying the asymptotic…
Various optimal gradient-based algorithms have been developed for smooth nonconvex optimization. However, many nonconvex machine learning problems do not belong to the class of smooth functions and therefore the existing algorithms are…
In this paper we are concerned with the solutions of the differential equation $f'''+ff''+g(f')=0$ on $[0,\infty)$, satisfying the boundary conditions $f(0)=\alpha$, $f'(0)=\beta\geq 0$, $f'(\infty)=\l$, and where $g$ is some given…
We consider the unconstrained optimization problem whose objective function is composed of a smooth and a non-smooth conponents where the smooth component is the expectation a random function. This type of problem arises in some interesting…
The article is devoted to the investigation of smoothness of functions $f(x_1,...,x_m)$ of variables $x_1,...,x_m$ in infinite fields with non-trivial multiplicative ultra-norms, where $m\ge 2$. Theorems about classes of smoothness $C^n$ or…
We provide new gradient-based methods for efficiently solving a broad class of ill-conditioned optimization problems. We consider the problem of minimizing a function $f : \mathbb{R}^d \rightarrow \mathbb{R}$ which is implicitly…
We consider gradient flow/gradient descent and heavy ball/accelerated gradient descent optimization for convex objective functions. In the gradient flow case, we prove the following: 1. If $f$ does not have a minimizer, the convergence…
This paper considers a local and non-local problem characterized by singular nonlinearity and a source term. Specifically, we focus on the following problem: \begin{equation}\label{A}\tag{P} -\Delta_{p} u + (-\Delta)^{s}_{q} u = f(x)…
We study the set of continuous functions that admit no spurious local optima (i.e. local minima that are not global minima) which we term \textit{global functions}. They satisfy various powerful properties for analyzing nonconvex and…
The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finite-sum smooth nonconvex objective function $F(w)=\frac{1}{n}…
We study the classical optimization problem $\min_{x \in \mathbb{R}^d} f(x)$ and analyze the gradient descent (GD) method in both nonconvex and convex settings. It is well-known that, under the $L$-smoothness assumption ($\|\nabla^2 f(x)\|…
Image smoothing is a fundamental procedure in applications of both computer vision and graphics. The required smoothing properties can be different or even contradictive among different tasks. Nevertheless, the inherent smoothing nature of…
Let $\mathbf{x}_j = \mathbf{\theta} + \mathbf{\epsilon}_j$, $j=1,\dots,n$ be i.i.d. copies of a Gaussian random vector $\mathbf{x}\sim\mathcal{N}(\mathbf{\theta},\mathbf{\Sigma})$ with unknown mean $\mathbf{\theta} \in \mathbb{R}^d$ and…
We introduce prox-convex for minimizing $F(x)=g(x)+h(C(x))+s(R(x))$, where $g$ and $h$ are convex, $C$ and $s$ are smooth, and each component of $R$ is convex (possibly nonsmooth). Here $g$ captures general convex objectives and indicator…
In this article we propose a new approach to an analysis of DC optimization problems. This approach was largely inspired by codifferential calculus and the method of codifferential descent and is based on the use of a so-called affine…
Numerical solutions of differential equations are usually not smooth functions. However, they should resemble the smoothness of the corresponding real solutions in one way or another. In two of our recent papers, a kind of spacial…
Over the past decades, the concept "partial smoothness" has been playing as a powerful tool in several fields involving nonsmooth analysis, such as nonsmooth optimization, inverse problems and operation research, etc. The essence of partial…
We study the complexity of approximating integrals of smooth functions at absolute precision $\varepsilon > 0$ with confidence level $1 - \delta \in (0,1)$. The optimal error rate for multivariate functions from classical isotropic Sobolev…