Related papers: Rigorous estimates for the relegation algorithm
We show that the Hedge algorithm, a method that is widely used in Machine Learning, can be interpreted as a particular instance of Dual Averaging schemes, which have recently been introduced by Nesterov for regret minimization. Based on…
Convergence analysis of consensus algorithms is revisited in the light of the Hilbert distance. Tsitsiklis Lyapunov function is shown to be the Hilbert distance to consensus in log coordinates. Birkhoff theorem, which proves contraction of…
This letter presents an almost sure convergence of the zeroth-order mirror descent algorithm. The algorithm admits non-smooth convex functions and a biased oracle which only provides noisy function value at any desired point. We approximate…
In this paper, we investigate the power of {\it regularization}, a common technique in reinforcement learning and optimization, in solving extensive-form games (EFGs). We propose a series of new algorithms based on regularizing the payoff…
The problem of stability of the triangular libration points in the planar circular restricted three-body problem is considered. A software package, intended for normalization of autonomous Hamiltonian systems by means of computer algebra,…
This paper deals with an improvement of the "a-priori stability bounds" on the variation of the action variables and on the stability time obtained from a given Birkhoff normal form around the elliptic equilibrium point of an Hamiltonian…
It has been the standard teaching of today that backward stability analysis is taught as absolute, just as in Newtonian physics time is taught absolute time. We will prove it is not true in general. It depends on algorithms. We will prove…
Tikhonov regularization involves minimizing the combination of a data discrepancy term and a regularizing term, and is the standard approach for solving inverse problems. The use of non-convex regularizers, such as those defined by trained…
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. However, these algorithms might perform poorly especially if…
We present a new modified Burg-Like algorithm for spectral estimation and adaptive signal processing, that yield the same prediction coefficients given by the Levinson algorithm for the solution of the normal equations. An equivalency proof…
Online learning algorithms are fast, memory-efficient, easy to implement, and applicable to many prediction problems, including classification, regression, and ranking. Several online algorithms were proposed in the past few decades, some…
The aim of this short paper is to show that some assumptions in [10] can be relaxed and even dropped when looking for weak solutions instead of strong ones. This improvement is a consequence of two results concerning gradient terms: an…
The standard approach for dealing with the ill-posedness of the training problem in machine learning and/or the reconstruction of a signal from a limited number of measurements is regularization. The method is applicable whenever the…
This paper presents an error analysis of classical and learned Tikhonov regularization schemes for inverse problems. We first demonstrate, both theoretically and numerically, that using a fixed regularization parameter across varying noise…
We present a new perspective on the popular Sinkhorn algorithm, showing that it can be seen as a Bregman gradient descent (mirror descent) of a relative entropy (Kullback-Leibler divergence). This viewpoint implies a new sublinear…
We study a stability property of probability laws with respect to small violations of algorithmic randomness. A sufficient condition of stability is presented in terms of Schnorr tests of algorithmic randomness. Most probability laws, like…
Leveraging algorithmic stability to derive sharp generalization bounds is a classic and powerful approach in learning theory. Since Vapnik and Chervonenkis [1974] first formalized the idea for analyzing SVMs, it has been utilized to study…
For perturbations of integrable Hamiltonians systems, the Nekhoroshev theorem shows that all solutions are stable for an exponentially long interval of time, provided the integrable part satisfies a steepness condition and the system is…
These lecture notes for a graduate class present the regularization theory for linear and nonlinear ill-posed operator equations in Hilbert spaces. Covered are the general framework of regularization methods and their analysis via spectral…
This report contains a numerical stability analysis of factorization algorithms for computing the Cholesky decomposition of symmetric positive definite matrices of displacement rank 2. The algorithms in the class can be expressed as…