相关论文: Gradient descent in a generalised Bregman distance…

We propose a variant of the approximate Bregman proximal gradient (ABPG) algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function. ABPG is known to converge globally to a stationary point even when the…

A discretization of a continuum theory with constraints or conserved quantities is called mimetic if it mirrors the conserved laws or constraints of the continuum theory at the discrete level. Such discretizations have been found useful in…

Constrained competitive optimization involves multiple agents trying to minimize conflicting objectives, subject to constraints. This is a highly expressive modeling language that subsumes most of modern machine learning. In this work we…

In this paper we introduce a novel method of gradient normalization and decay with respect to depth. Our method leverages the simple concept of normalizing all gradients in a deep neural network, and then decaying said gradients with…

We consider regularization of non-convex optimization problems involving a non-linear least-squares objective. By adding an auxiliary set of variables, we introduce a novel regularization framework whose corresponding objective function is…

The Bregman-Wasserstein divergence is the optimal transport cost when the underlying cost function is given by a Bregman divergence, and arises naturally in fields such as statistics and machine learning. We establish fundamental properties…

The aim of this paper is to present the convergence analysis of a very general class of gradient projection methods for smooth, constrained, possibly nonconvex, optimization. The key features of these methods are the Armijo linesearch along…

Minimization of suitable statistical distances~(between the data and model densities) has proved to be a very useful technique in the field of robust inference. Apart from the class of $\phi$-divergences of \cite{a} and \cite{b}, the…

Graded Lagrangian formalism in terms of a Grassmann-graded variational bicomplex on graded manifolds is developed in a very general setting. This formalism provides the comprehensive description of reducible degenerate Lagrangian systems,…

Deep learning has aroused extensive attention due to its great empirical success. The efficiency of the block coordinate descent (BCD) methods has been recently demonstrated in deep neural network (DNN) training. However, theoretical…

We propose a learning framework based on stochastic Bregman iterations, also known as mirror descent, to train sparse neural networks with an inverse scale space approach. We derive a baseline algorithm called LinBreg, an accelerated…

We develop a new proximal-gradient method for minimizing the sum of a differentiable, possibly nonconvex, function plus a convex, possibly non differentiable, function. The key features of the proposed method are the definition of a…

This paper introduces the $(\alpha, \Gamma)$-descent, an iterative algorithm which operates on measures and performs $\alpha$-divergence minimisation in a Bayesian framework. This gradient-based procedure extends the commonly-used…

Distributed optimization aims to optimize a global objective formed by a sum of coupled local convex functions over a graph via only local computation and communication. In this paper, we propose the Bregman parallel direction method of…

Inspired by the recent paper (L. Ying, Mirror descent algorithms for minimizing interacting free energy, Journal of Scientific Computing, 84 (2020), pp. 1-14),we explore the relationship between the mirror descent and the variable metric…

An analogue of the total variation prior for the normal vector field along the boundary of piecewise flat shapes in 3D is introduced. A major class of examples are triangulated surfaces as they occur for instance in finite element…

We propose and analyze a variant of the classic Polyak-Ruppert averaging scheme, broadly used in stochastic gradient methods. Rather than a uniform average of the iterates, we consider a weighted average, with weights decaying in a…

The aim of this paper is to introduce and study a two-step debiasing method for variational regularization. After solving the standard variational problem, the key idea is to add a consecutive debiasing step minimizing the data fidelity on…

In this paper, we consider a nonconvex optimization problem with nonlinear equality constraints. We assume that both, the objective function and the functional constraints are locally smooth. For solving this problem, we propose a…

Natural Gradient Descent, a second-degree optimization method motivated by the information geometry, makes use of the Fisher Information Matrix instead of the Hessian which is typically used. However, in many cases, the Fisher Information…