Related papers: Implicit bias of any algorithm: bounding bias via …

Stochastic gradient descent is one of the most common iterative algorithms used in machine learning and its convergence analysis is a rich area of research. Understanding its convergence properties can help inform what modifications of it…

We study first-order optimization algorithms for computing the barycenter of Gaussian distributions with respect to the optimal transport metric. Although the objective is geodesically non-convex, Riemannian GD empirically converges…

Implicit bias induced by gradient-based algorithms is essential to the generalization of overparameterized models, yet its mechanisms can be subtle. This work leverages the Normalized Steepest Descent} (NSD) framework to investigate how…

We propose and analyze a randomized zeroth-order approach based on approximating the exact gradient byfinite differences computed in a set of orthogonal random directions that changes with each iteration. A number ofpreviously proposed…

We consider minimizing a smooth function subject to a summation constraint over its variables. By exploiting a connection between the greedy 2-coordinate update for this problem and equality-constrained steepest descent in the 1-norm, we…

Learning algorithms for implicit generative models can optimize a variety of criteria that measure how the data distribution differs from the implicit model distribution, including the Wasserstein distance, the Energy distance, and the…

Given two points $p,q$ in the real plane, the signed area of the rectangle with the diagonal $[pq]$ equals the square of the Minkowski distance between the points $p,q$. We prove that $N>1$ points in the Minkowski plane $\R^{1,1}$ generate…

We state and prove a stabilisation result for solutions of abstract gradient systems associated with nonsmooth energy functions on infinite dimensional Hilbert spaces. One feature is that in this general setting the assumption on the range…

We investigate an inertial algorithm of gradient type in connection with the minimization of a nonconvex differentiable function. The algorithm is formulated in the spirit of Nesterov's accelerated convex gradient method. We show that the…

We study first order methods to compute the barycenter of a probability distribution $P$ over the space of probability measures with finite second moment. We develop a framework to derive global rates of convergence for both gradient…

We show that independent and uniformly distributed sampling points are as good as optimal sampling points for the approximation of functions from the Sobolev space $W_p^s(\Omega)$ on bounded convex domains $\Omega\subset \mathbb{R}^d$ in…

In this paper, the problem of finding optimal success probabilities of static linear optics quantum gates is linked to the theory of convex optimization. It is shown that by exploiting this link, upper bounds for the success probability of…

We study nonparametric change-point estimation from indirect noisy observations. Focusing on the white noise convolution model, we consider two classes of functions that are smooth apart from the change-point. We establish lower bounds on…

Let \Theta be a smooth compact oriented manifold without boundary, embedded in a euclidean space and let \gamma be a smooth map \Theta into a riemannian manifold \Lambda. An unknown state \theta \in \Theta is observed via X=\theta+\epsilon…

Change-plane regression identifies subpopulations through an interpretable linear threshold rule, but likelihood-based inference for the hard-threshold boundary is nonregular: objectives are non-smooth, the boundary is weakly identified…

In this paper we propose two proximal gradient algorithms for fractional programming problems in real Hilbert spaces, where the numerator is a proper, convex and lower semicontinuous function and the denominator is a smooth function, either…

The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C1-smooth functions with Lipschitzian gradients, i.e., for problems of C1,1 optimization. We show that the sequence of gradients generated by IGD converges…

We propose a group theoretical method to study Isgur-Wise functions. A current matrix element splits into a heavy quark matrix element and an overlap of the initial and final clouds, related to the IW functions, that contain the long…

By the continuous mapping theorem, if a sequence of $d$-dimensional random vectors $(\mathbf{W}_n)_{n\geq1}$ converges in distribution to a multivariate normal random variable $\Sigma^{1/2}\mathbf{Z}$, then the sequence of random variables…

We optimize the running time of the primal-dual algorithms by optimizing their stopping criteria for solving convex optimization problems under affine equality constraints, which means terminating the algorithm earlier with fewer…