Related papers: Rigorous estimates for the relegation algorithm
Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by…
We use the uniform semiclassical approximation in order to derive the fidelity decay in the regime of large perturbations. Numerical computations are presented which agree with our theoretical predictions. Moreover, our theory allows to…
We consider the setting of online logistic regression and consider the regret with respect to the 2-ball of radius B. It is known (see [Hazan et al., 2014]) that any proper algorithm which has logarithmic regret in the number of samples…
This paper provides error analyses of the algorithms most commonly used for the evaluation of the Chebyshev polynomial of the first kind $T_N(x)$. Some of these algorithms are shown to be backward stable. This means that the computed value…
Low-rank regularization (LRR) has been widely applied in various machine learning tasks, but the associated optimization is challenging. Directly optimizing the rank function under constraints is NP-hard in general. To overcome this…
This paper presents a detailed theoretical analysis of the three stochastic approximation proximal gradient algorithms proposed in our companion paper [49] to set regularization parameters by marginal maximum likelihood estimation. We prove…
We present a subgradient method for minimizing non-smooth, non-Lipschitz convex optimization problems. The only structure assumed is that a strictly feasible point is known. We extend the work of Renegar [5] by taking a different…
Steepness is a geometric property which, together with complex-analyticity, is needed in order to insure stability of a near-integrable hamiltonian system over exponentially long times. Following a strategy developed by Nekhoro-shev, we…
We prove a $C^\infty$ version of the Nekhoroshev's estimate on the stability times of the actions in close to integrable Hamiltonian systems. The proof we give is a variant of the original Nekhoroshev's proof and it consists in first…
Conditional stability estimates require additional regularization for obtaining stable approximate solutions if the validity area of such estimates is not completely known. In this context, we consider ill-posed nonlinear inverse problems…
The overall performance or expected excess risk of an iterative machine learning algorithm can be decomposed into training error and generalization error. While the former is controlled by its convergence analysis, the latter can be tightly…
We generalize the Safe Extremum Seeking algorithm to address the minimization of an unknown objective function subject to multiple unknown inequality and equality constraints, relying on recent results of gradient flow systems. These…
The classical Eagleson's theorem states that if appropriately normalized Birkhoff sums generated by a measurable function and a probability preserving transformation converge in distribution, then they also converge in distribution with…
This paper aims at developing two versions of the generalized Newton method to compute not merely arbitrary local minimizers of nonsmooth optimization problems but just those, which possess an important stability property known as tilt…
Likelihood ratios are used for a variety of applications in particle physics data analysis, including parameter estimation, unfolding, and anomaly detection. When the data are high-dimensional, neural networks provide an effective tools for…
In this paper, we address the efficient implementation of moving horizon state estimation of constrained discrete-time linear systems. We propose a novel iteration scheme which employs a proximity-based formulation of the underlying…
We present a new perspective on the celebrated Sinkhorn algorithm by showing that is a special case of incremental/stochastic mirror descent. In order to see this, one should simply plug Kullback-Leibler divergence in both mirror map and…
In this paper, we study the online learning algorithm without explicit regularization terms. This algorithm is essentially a stochastic gradient descent scheme in a reproducing kernel Hilbert space (RKHS). The polynomially decaying step…
This paper is about the surprising interaction of a foundational result from model theory, about stability of theories, with algorithmic stability in learning. First, in response to gaps in existing learning models, we introduce a new…
The inverse-free extreme learning machine (ELM) algorithm proposed in [4] was based on an inverse-free algorithm to compute the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of…