Related papers: Scalable Control Variates for Monte Carlo Methods …

In this paper a novel modification of the multilevel Monte Carlo approach, allowing for further significant complexity reduction, is proposed. The idea of the modification is to use the method of control variates to reduce variance at level…

We investigate Monte Carlo based algorithms for solving stochastic control problems with probabilistic constraints. Our motivation comes from microgrid management, where the controller tries to optimally dispatch a diesel generator while…

Standard Monte Carlo computation is widely known to exhibit a canonical square-root convergence speed in terms of sample size. Two recent techniques, one based on control variate and one on importance sampling, both derived from an…

We propose neural control variates (NCV) for unbiased variance reduction in parametric Monte Carlo integration. So far, the core challenge of applying the method of control variates has been finding a good approximation of the integrand…

A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible MCMC samplers. We propose the use of a specific class of functions as control…

Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by…

It is well known that Monte Carlo integration with variance reduction by means of control variates can be implemented by the ordinary least squares estimator for the intercept in a multiple linear regression model. A central limit theorem…

We use classical tools from calculus of variations to formally derive necessary conditions for a Markov control to be optimal in a standard finite time horizon stochastic control problem. As an example, we solve the well-known Merton…

We present a multilevel stochastic gradient descent method for the optimal control of systems governed by partial differential equations under uncertain input data. The gradient descent method used to find the optimal control leverages a…

Adaptive Monte Carlo methods are very efficient techniques designed to tune simulation estimators on-line. In this work, we present an alternative to stochastic approximation to tune the optimal change of measure in the context of…

Model predictive control (MPC) schemes have a proven track record for delivering aggressive and robust performance in many challenging control tasks, coping with nonlinear system dynamics, constraints, and observational noise. Despite their…

The problem of optimising functions with intractable gradients frequently arise in machine learning and statistics, ranging from maximum marginal likelihood estimation procedures to fine-tuning of generative models. Stochastic approximation…

Neural control variates (NCVs) have emerged as a powerful tool for variance reduction in Monte Carlo (MC) simulations, particularly in high-dimensional problems where traditional control variates are difficult to construct analytically. By…

We introduce a variational algorithm to estimate the likelihood of a rare event within a nonequilibrium molecular dynamics simulation through the evaluation of an optimal control force. Optimization of a control force within a chosen basis…

Policy gradient methods have demonstrated success in reinforcement learning tasks that have high-dimensional continuous state and action spaces. However, policy gradient methods are also notoriously sample inefficient. This can be…

Reliable robotic manipulation requires control policies that can accurately represent and adapt to uncertainty arising from contact-rich interactions. Modern data-driven methods mitigate uncertainty through large-scale training and…

We present a general approach to greatly increase at little cost the efficiency of Monte Carlo algorithms. To each observable to be computed we associate a renormalized observable (improved estimator) having the same average but a different…

Monte Carlo methods are widely used for neutron transport simulations at least partly because of the accuracy they bring to the modeling of these problems. However, the computational burden associated with the slow convergence rate of Monte…

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stochastic gradients…

In the day-to-day operation of a power system, the system operator repeatedly solves short-term generation planning problems. When formulating these problems the operators have to weigh the risk of costly failures against increased…