Related papers: Tuning diagonal scale matrices for HMC

Isomorphs are curves in the thermodynamic phase diagram along which structure and dynamics are invariant to a good approximation. There are two main ways to trace out isomorphs, the configurational-adiabat method and the…

The generalized linear mixed model (GLMM) is widely used for analyzing correlated data, particularly in large-scale biomedical and social science applications. Scalable Bayesian inference for GLMMs is challenging because the marginal…

The plain stochastic gradient descent and momentum stochastic gradient descent have extremely wide applications in deep learning due to their simple settings and low computational complexity. The momentum stochastic gradient descent uses…

We propose a stochastic modified equations (SME) for modeling the asynchronous stochastic gradient descent (ASGD) algorithms. The resulting SME of Langevin type extracts more information about the ASGD dynamics and elucidates the…

In many applications involving large dataset or online updating, stochastic gradient descent (SGD) provides a scalable way to compute parameter estimates and has gained increasing popularity due to its numerical convenience and memory…

Hamiltonian Monte Carlo (HMC) is an efficient method of simulating smooth distributions and has motivated the widely used No-U-turn Sampler (NUTS) and software Stan. We build on NUTS and the technique of "unbiased sampling" to design HMC…

The pseudo-marginal algorithm is a variant of the Metropolis--Hastings algorithm which samples asymptotically from a probability distribution when it is only possible to estimate unbiasedly an unnormalized version of its density.…

Growing heterogeneity and configurability in HPC architectures has made auto-tuning applications and runtime parameters on these systems very complex. Users are presented with a multitude of options to configure parameters. In addition to…

Decentralized stochastic gradient method emerges as a promising solution for solving large-scale machine learning problems. This paper studies the decentralized Markov chain gradient descent (DMGD) algorithm - a variant of the decentralized…

Hamiltonian Monte Carlo (HMC) is a state-of-the-art Markov chain Monte Carlo sampling algorithm for drawing samples from smooth probability densities over continuous spaces. We study the variant most widely used in practice, Metropolized…

Sequential Monte Carlo (SMC) methods offer a principled approach to Bayesian uncertainty quantification but are traditionally limited by the need for full-batch gradient evaluations. We introduce a scalable variant by incorporating…

We propose and analyze an iterative high-order hybridized discontinuous Galerkin (iHDG) discretization for linear partial differential equations. We improve our previous work (SIAM J. Sci. Comput. Vol. 39, No. 5, pp. S782--S808) in several…

We introduce a finite element method for numerical upscaling of second order elliptic equations with highly heterogeneous coefficients. The method is based on a mixed formulation of the problem and the concepts of the domain decomposition…

One of the most common methods to train machine learning algorithms today is the stochastic gradient descent (SGD). In a distributed setting, SGD-based algorithms have been shown to converge theoretically under specific circumstances. A…

Simultaneous matrix diagonalization is used as a subroutine in many machine learning problems, including blind source separation and paramater estimation in latent variable models. Here, we extend algorithms for performing joint…

How heterogeneous multiscale methods (HMM) handle fluctuations acting on the slow variables in fast-slow systems is investigated. In particular, it is shown via analysis of central limit theorems (CLT) and large deviation principles (LDP)…

Sequential Monte Carlo (SMC) methods are not only a popular tool in the analysis of state space models, but offer an alternative to MCMC in situations where Bayesian inference must proceed via simulation. This paper introduces a new SMC…

In [ESAIM: M2AN, 54(2020), 2229-2264], we proposed an HDG method to approximate the solution of a tangential boundary control problem for the Stokes equations and obtained an optimal convergence rate for the optimal control {that reflects…

When approximating the expectations of a functional of a solution to a stochastic differential equation, the numerical performance of deterministic quadrature methods, such as sparse grid quadrature and quasi-Monte Carlo (QMC) methods, may…

Stochastic gradient descent method and its variants constitute the core optimization algorithms that achieve good convergence rates for solving machine learning problems. These rates are obtained especially when these algorithms are…