相关论文: Understanding and Improving the Wang-Landau Algori…

This paper proposes a novel analysis for the Scaffold algorithm, a popular method for dealing with data heterogeneity in federated learning. While its convergence in deterministic settings--where local control variates mitigate client…

Studying sample path behaviour of stochastic fields/processes is a classical research topic in probability theory and related areas such as fractal geometry. To this end, many methods have been developed since a long time in Gaussian…

Wave-front sensing from focal plane multiple images is a promising technique for high-contrast imaging systems. However, the wave-front error of an optics system can be properly reconstructed only when it is very small. This paper presents…

We explain the fundamental challenges of sampling from multimodal distributions, particularly for high-dimensional problems. We present the major types of MCMC algorithms that are designed for this purpose, including parallel tempering,…

The Coordinate Ascent Variational Inference scheme is a popular algorithm used to compute the mean-field approximation of a probability distribution of interest. We analyze its random scan version, under log-concavity assumptions on the…

We propose a new discrete-time online parameter estimation algorithm that combines two different aspects, one that adds momentum, and another that includes a time-varying learning rate. It is well known that recursive least squares based…

In latent-position random graph models (LPMs), latent vertex positions $U_{1},\ldots,U_{n}$ are sampled from some distribution on a latent space $\Omega$, then edges of an observed graph $G = ([n],E)$ are sampled with some probability…

Gaussian mixtures are commonly used for modeling heavy-tailed error distributions in robust linear regression. Combining the likelihood of a multivariate robust linear regression model with a standard improper prior distribution yields an…

The systematization of the purely Lagrangean approach to constrained systems in the form of an algorithm involves the iterative construction of a generalized Hessian matrix W taking a rectangular form. This Hessian will exhibit as many left…

Local general depth ($LGD$) functions are used for describing the local geometric features and mode(s) in multivariate distributions. In this paper, we undertake a rigorous systematic study of $LGD$ and establish several analytical and…

Hyperparameter tuning is a common technique for improving the performance of neural networks. Most techniques for hyperparameter search involve an iterated process where the model is retrained at every iteration. However, the expected…

Stochastic saddle point (SSP) problems are, in general, less studied compared to stochastic minimization problems. However, SSP problems emerge from machine learning (adversarial training, e.g., GAN, AUC maximization), statistics (robust…

This paper proposes a proof of the convergence of a distributed and asynchronous version of the Kiefer-Wolfowitz algorithm.

We study the convergence rate of randomly truncated stochastic algorithms, which consist in the truncation of the standard Robbins-Monro procedure on an increasing sequence of compact sets. Such a truncation is often required in practice to…

We study the convergence rate of randomly truncated stochastic algorithms, which consist in the truncation of the standard Robbins-Monro procedure on an increasing sequence of compact sets. Such a truncation is often required in practice to…

Within the framework of weighted integrable Hamiltonian systems, we study the long-time behavior of the associated statistical ensembles. We construct an action-dependent angular conjugacy that rectifies the nonuniform angular flow into a…

Projection-free optimization via different variants of the Frank-Wolfe (FW) method has become one of the cornerstones in large scale optimization for machine learning and computational statistics. Numerous applications within these fields…

In maximum-likelihood quantum state tomography, both the sample size and dimension grow exponentially with the number of qubits. It is therefore desirable to develop a stochastic first-order method, just like stochastic gradient descent for…

LLM alignment ensures that large language models behave safely and effectively by aligning their outputs with human values, goals, and intentions. Aligning LLMs employ huge amounts of data, computation, and time. Moreover, curating data…

We consider the minimization of a sum of a smooth function with a nonsmooth composite function, where the composition is applied on a random linear mapping. This random composite model encompasses many problems, and can especially capture…