Related papers: Randomized approximation of summable sequences -- …

The EM algorithm is a powerful tool for maximum likelihood estimation with missing data. In practice, the calculations required for the EM algorithm are often intractable. We review numerous methods to circumvent this intractability, all of…

While mixture of linear regressions (MLR) is a well-studied topic, prior works usually do not analyze such models for prediction error. In fact, {\em prediction} and {\em loss} are not well-defined in the context of mixtures. In this paper,…

The celebrated Monte Carlo method estimates an expensive-to-compute quantity by random sampling. Bandit-based Monte Carlo optimization is a general technique for computing the minimum of many such expensive-to-compute quantities by adaptive…

The estimation of normalizing constants is a fundamental step in probabilistic model comparison. Sequential Monte Carlo methods may be used for this task and have the advantage of being inherently parallelizable. However, the standard…

We study a version of the proximal gradient algorithm for which the gradient is intractable and is approximated by Monte Carlo methods (and in particular Markov Chain Monte Carlo). We derive conditions on the step size and the Monte Carlo…

We examine a wide class of stochastic approximation algorithms for solving (stochastic) nonlinear problems on Riemannian manifolds. Such algorithms arise naturally in the study of Riemannian optimization, game theory and optimal transport,…

We consider the order of convergence for linear and nonlinear Monte Carlo approximation of compact embeddings from Sobolev spaces of dominating mixed smoothness defined on the torus $\mathbb{T}^d$ into the space $L_{\infty}(\mathbb{T}^d)$…

While multilevel Monte Carlo (MLMC) methods for the numerical approximation of partial differential equations with random coefficients enjoy great popularity, combinations with spatial adaptivity seem to be rare. We present an adaptive MLMC…

This work proposes an adaptive sequential Monte Carlo sampling algorithm to solve Bayesian inverse problems in scenarios where likelihood evaluations are costly but can be approximated using a surrogate model built from previous evaluations…

We propose novel randomized optimization methods for high-dimensional convex problems based on restrictions of variables to random subspaces. We consider oblivious and data-adaptive subspaces and study their approximation properties via…

In solving simulation-based stochastic root-finding or optimization problems that involve rare events, such as in extreme quantile estimation, running crude Monte Carlo can be prohibitively inefficient. To address this issue, importance…

The Stochastic Approximation EM (SAEM) algorithm, a variant stochastic approximation of EM, is a versatile tool for inference in incomplete data models. In this paper, we review the fundamental EM algorithm and then focus especially on the…

We propose an adaptive importance sampling scheme for Gaussian approximations of intractable posteriors. Optimization-based approximations like variational inference can be too inaccurate while existing Monte Carlo methods can be too slow.…

We focus the use of \emph{row sampling} for approximating matrix algorithms. We give applications to matrix multipication; sparse matrix reconstruction; and, \math{\ell_2} regression. For a matrix \math{\matA\in\R^{m\times d}} which…

This paper provides sufficient conditions over the sequence of samples and parameters of an adaptive Markov Chain Monte Carlo (MCMC) algorithm to ensure ergodicity with respect to a target distribution that can have unbounded support. These…

Optimizing machine learning algorithms that are used to solve the objective function has been of great interest. Several approaches to optimize common algorithms, such as gradient descent and stochastic gradient descent, were explored. One…

In large-data applications, such as the inference process of diffusion models, it is desirable to design sampling algorithms with a high degree of parallelization. In this work, we study the adaptive complexity of sampling, which is the…

A Monte Carlo algorithm is said to be adaptive if it automatically calibrates its current proposal distribution using past simulations. The choice of the parametric family that defines the set of proposal distributions is critical for good…

We investigate lower bounds on the subgeometric convergence of adaptive Markov chain Monte Carlo under any adaptation strategy. In particular, we prove general lower bounds in total variation and on the weak convergence rate under general…

We construct Monte Carlo methods for the $L^2$-approximation in Hilbert spaces of multivariate functions sampling no more than $n$ function values of the target function. Their errors catch up with the rate of convergence and the…