Related papers: Local Optimality and Generalization Guarantees for…

Empirical Risk Minimization (ERM) based machine learning algorithms have suffered from weak generalization performance on data obtained from out-of-distribution (OOD). To address this problem, Invariant Risk Minimization (IRM) objective was…

Parameter estimation in Markov random fields (MRFs) is a difficult task, in which inference over the network is run in the inner loop of a gradient descent procedure. Replacing exact inference with approximate methods such as loopy belief…

This paper considers the optimal scaling problem for high-dimensional random walk Metropolis algorithms for densities which are differentiable in Lp mean but which may be irregular at some points (like the Laplace density for example)…

We consider the Random Walk Metropolis algorithm on $\mathbb{R}^n$ with Gaussian proposals, and when the target probability measure is the $n$-fold product of a one-dimensional law. It is well known (see Roberts et al. (Ann. Appl. Probab. 7…

In molecular dynamics, several algorithms have been designed over the past few years to accelerate the exit event from a metastable region of the configuration space. Some of them are based on the fact that the exit event from a metastable…

Among random sampling methods, Markov Chain Monte Carlo algorithms are foremost. Using a combination of analytical and numerical approaches, we study their convergence properties towards the steady state, within a random walk Metropolis…

In stochastic convex optimization the goal is to minimize a convex function $F(x) \doteq {\mathbf E}_{{\mathbf f}\sim D}[{\mathbf f}(x)]$ over a convex set $\cal K \subset {\mathbb R}^d$ where $D$ is some unknown distribution and each…

The Metropolis-adjusted Langevin algorithm (MALA) is a Metropolis-Hastings method for approximate sampling from continuous distributions. We derive upper bounds for the contraction rate in Kantorovich-Rubinstein-Wasserstein distance of the…

Langevin algorithms are gradient descent methods augmented with additive noise, and are widely used in Markov Chain Monte Carlo (MCMC) sampling, optimization, and machine learning. In recent years, the non-asymptotic analysis of Langevin…

The $\ell_0$-constrained empirical risk minimization ($\ell_0$-ERM) is a promising tool for high-dimensional statistical estimation. The existing analysis of $\ell_0$-ERM estimator is mostly on parameter estimation and support recovery…

Since Kramers' pioneering work in 1940, significant efforts have been devoted to studying Langevin equations applied to physical and chemical reactions projected onto few collective variables, with particular focus on the inference of their…

Stochastic approximation methods play a central role in maximum likelihood estimation problems involving intractable likelihood functions, such as marginal likelihoods arising in problems with missing or incomplete data, and in parametric…

Recently there is a large amount of work devoted to the study of Markov chain stochastic gradient methods (MC-SGMs) which mainly focus on their convergence analysis for solving minimization problems. In this paper, we provide a…

As written by statistician George Box "All models are wrong, but some are useful", standard diffusion derivation or Feynman path ensembles use nonphysical infinite velocity/kinetic energy nowhere differentiable trajectories - what seems…

In many estimation problems, e.g. linear and logistic regression, we wish to minimize an unknown objective given only unbiased samples of the objective function. Furthermore, we aim to achieve this using as few samples as possible. In the…

We consider the problem of sampling from a target distribution, which is \emph {not necessarily logconcave}, in the context of empirical risk minimization and stochastic optimization as presented in Raginsky et al. (2017). Non-asymptotic…

The Expectation-Maximization (EM) algorithm is an iterative method to maximize the log-likelihood function for parameter estimation. Previous works on the convergence analysis of the EM algorithm have established results on the asymptotic…

We study the sample complexity of the best-case Empirical Risk Minimizer in the setting of stochastic convex optimization. We show that there exists an instance in which the sample size is linear in the dimension, learning is possible, but…

Many systems across the sciences evolve through a combination of multiplicative growth and diffusive transport. In the presence of disorder, these systems tend to form localized structures which alternate between long periods of relative…

It is well known that Empirical Risk Minimization (ERM) may attain minimax suboptimal rates in terms of the mean squared error (Birg\'e and Massart, 1993). In this paper, we prove that, under relatively mild assumptions, the suboptimality…