Related papers: Large and moderate deviation principles for averag…
Importance sampling is a popular method for efficient computation of various properties of a distribution such as probabilities, expectations, quantiles etc. The output of an importance sampling algorithm can be represented as a weighted…
We consider a class of nonparametric time series regression models in which the regressor takes values in a sequence space. Technical challenges that hampered theoretical advances in these models include the lack of associated Lebesgue…
For min-max optimization and variational inequalities problems (VIP) encountered in diverse machine learning tasks, Stochastic Extragradient (SEG) and Stochastic Gradient Descent Ascent (SGDA) have emerged as preeminent algorithms. Constant…
This paper introduces an iterative algorithm for training nonparametric additive models that enjoys favorable memory storage and computational requirements. The algorithm can be viewed as the functional counterpart of stochastic gradient…
The first aim of this paper is to establish the weak convergence rate of nonlinear two-time-scale stochastic approximation algorithms. Its second aim is to introduce the averaging principle in the context of two-time-scale stochastic…
The goal of this paper is to study the Moderate Deviation Principle (MDP) for a system of stochastic reaction-diffusion equations with a time-scale separation in slow and fast components and small noise in the slow component. Based on weak…
We investigate statistical properties of the optimal value of the Sample Average Approximation of stochastic programs, continuing the study in Kr\"atschmer (2023). Central Limit Theorem type results are derived for the optimal value. As a…
Given an $n$-dimensional random vector $X^{(n)}$ , for $k < n$, consider its $k$-dimensional projection $\mathbf{a}_{n,k}X^{(n)}$, where $\mathbf{a}_{n,k}$ is an $n \times k$-dimensional matrix belonging to the Stiefel manifold…
Stochastic Gradient Descent (SGD) and its variants are the most used algorithms in machine learning applications. In particular, SGD with adaptive learning rates and momentum is the industry standard to train deep networks. Despite the…
There are several applications of stochastic optimization where one can benefit from a robust estimate of the gradient. For example, domains such as distributed learning with corrupted nodes, the presence of large outliers in the training…
In this paper an easy to implement method of stochastically weighing short and long memory linear processes is introduced. The method renders asymptotically exact size confidence intervals for the population mean which are significantly…
We propose a new estimator for the high-dimensional linear regression model with observation error in the design where the number of coefficients is potentially larger than the sample size. The main novelty of our procedure is that the…
In this paper, we refine the Berry-Esseen bounds for the multivariate normal approximation of Polyak-Ruppert averaged iterates arising from the linear stochastic approximation (LSA) algorithm with decreasing step size. We consider the…
We consider the moderate deviations behaviors for two (co-) volatility estima-tors: generalised bipower variation, Hayashi-Yoshida estimator. The results are obtained by using a new result about the moderate deviations principle for…
We study the averaging-based distributed optimization solvers over random networks. We show a general result on the convergence of such schemes using weight-matrices that are row-stochastic almost surely and column-stochastic in expectation…
We present a new algorithm for approximate inference in probabilistic programs, based on a stochastic gradient for variational programs. This method is efficient without restrictions on the probabilistic program; it is particularly…
The paper considers distributed stochastic optimization over randomly switching networks, where agents collaboratively minimize the average of all agents' local expectation-valued convex cost functions. Due to the stochasticity in gradient…
We consider a generic empirical composition optimization problem, where there are empirical averages present both outside and inside nonlinear loss functions. Such a problem is of interest in various machine learning applications, and…
The Adaptive Multilevel Splitting (AMS) algorithm is a powerful and versatile method for the simulation of rare events. It is based on an interacting (via a mutation-selection procedure) system of replicas, and depends on two integer…
Majorization-minimization algorithms consist of iteratively minimizing a majorizing surrogate of an objective function. Because of its simplicity and its wide applicability, this principle has been very popular in statistics and in signal…