相关论文: On the gradient rearrangement of functions
In this paper we consider the variable inequality problem, that is, to find a solution of the inclusion given by the sum of a function and a point-to-cone application. This problem can be seen as a generalization of the classical system…
Convergence of the gradient descent algorithm has been attracting renewed interest due to its utility in deep learning applications. Even as multiple variants of gradient descent were proposed, the assumption that the gradient of the…
Several recent works have explored stochastic gradient methods for variational inference that exploit the geometry of the variational-parameter space. However, the theoretical properties of these methods are not well-understood and these…
We introduce the $\alpha,\beta$-symmetric difference derivative and the $\alpha,\beta$-symmetric N\"orlund sum. The associated symmetric quantum calculus is developed, which can be seen as a generalization of the forward and backward…
We prove gradient estimates for harmonic functions with respect to a $d$-dimensional unimodal pure-jump Levy process under some mild assumptions on the density of its Levy measure. These assumptions allow for a construction of an unimodal…
The article considers the discrete analogue of the method of quickest descent for an inverse Acoustics problem in case of a smooth source. The authors derived the gradient of functional in differential and discrete cases, described the…
We give a pedagogical introduction to static spherically symmetric solutions in models of New GR, both explaining the basics and showing how all such vacuum solutions can be obtained in elementary functions. In doing so, we coherently…
We consider the differentiation of the value function for parametric optimization problems. Such problems are ubiquitous in Machine Learning applications such as structured support vector machines, matrix factorization and min-min or…
In this paper we consider large-scale composite optimization problems having the objective function formed as a sum of two terms (possibly nonconvex), one has (block) coordinate-wise Lipschitz continuous gradient and the other is…
We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping. We assume the whole…
The problem of minimization of the sum of two convex functions has various theoretical and real-world applications. One of the popular methods for solving this problem is the proximal gradient method (proximal forward-backward algorithm). A…
The conjugate gradient method is a widely used algorithm for the numerical solution of a system of linear equations. It is particularly attractive because it allows one to take advantage of sparse matrices and produces (in case of infinite…
A stochastic conjugate gradient method for approximation of a function is proposed. The proposed method avoids computing and storing the covariance matrix in the normal equations for the least squares solution. In addition, the method…
In this paper we introduce a new type of symmetrization, which preserves the anisotropic perimeter of the level sets of a suitable concave smooth function, in order to prove sharp comparison results for solutions of a class of homogeneous…
We consider optimal control of an elliptic two-point boundary value problem governed by functions of bounded variation (BV). The cost functional is composed of a tracking term for the state and the BV-seminorm of the control. We use the…
We study the problem of minimizing a relatively-smooth convex function using stochastic Bregman gradient methods. We first prove the convergence of Bregman Stochastic Gradient Descent (BSGD) to a region that depends on the noise (magnitude…
We present a new class of gradient-type optimization methods that extends vanilla gradient descent, mirror descent, Riemannian gradient descent, and natural gradient descent. Our approach involves constructing a surrogate for the objective…
We study the asymptotic shape of the trajectory of the stochastic gradient descent algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit theorem for the properly rescaled…
We consider stochastic gradient descent algorithms for minimizing a non-smooth, strongly-convex function. Several forms of this algorithm, including suffix averaging, are known to achieve the optimal $O(1/T)$ convergence rate in…
Gradient schemes is a framework that enables the unified convergence analysis of many numerical methods for elliptic and parabolic partial differential equations: conforming and non-conforming Finite Element, Mixed Finite Element and Finite…