Related papers: Stochastic optimization on continuous domains with…
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a…
In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty…
Multi-stage stochastic optimization lies at the core of decision-making under uncertainty. As the analytical solution is available only in exceptional cases, dynamic optimization aims to efficiently find approximations but often neglects…
We develop a Monte-Carlo based numerical method for solving discrete-time stochastic optimal control problems with inventory. These are optimal control problems in which the control affects only a deterministically evolving inventory…
Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…
We investigate planning in time-critical domains represented as Markov Decision Processes, showing that search based techniques can be a very powerful method for finding close to optimal plans. To reduce the computational cost of planning…
In this paper, we propose a multilevel stochastic framework for the solution of nonconvex unconstrained optimization problems. The proposed approach uses random regularized first-order models that exploit an available hierarchical…
Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by…
Markov chain Monte Carlo is a class of algorithms for drawing Markovian samples from high-dimensional target densities to approximate the numerical integration associated with computing statistical expectation, especially in Bayesian…
First-order methods are often analyzed via their continuous-time models, where their worst-case convergence properties are usually approached via Lyapunov functions. In this work, we provide a systematic and principled approach to find and…
In recent years dynamical systems (of deterministic and stochastic nature), describing many models in mathematics, physics, engineering and finances, become more and more complex. Numerical analysis narrowed only to deterministic algorithms…
This article is a tutorial on Markov chain Monte Carlo simulations and their statistical analysis. The theoretical concepts are illustrated through many numerical assignments from the author's book on the subject. Computer code (in Fortran)…
In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas…
This paper studies a continuous-time market {under stochastic environment} where an agent, having specified an investment horizon and a target terminal mean return, seeks to minimize the variance of the return with multiple stocks and a…
In this paper we develop a method to compute the solution to a countable (finite or infinite) set of equations that occurs in many different fields including Markov processes that model queueing systems, birth-and-death processes and…
In dynamic Monte Carlo simulations, using for example the Metropolis dynamic, it is often required to simulate for long times and to simulate large systems. We present an overview of advanced algorithms to simulate for larger times and to…
Application domains of Bayesian optimization include optimizing black-box functions or very complex functions. The functions we are interested in describe complex real-world systems applied in industrial settings. Even though they do have…
Monte Carlo (MC) sampling methods are widely applied in Bayesian inference, system simulation and optimization problems. The Markov Chain Monte Carlo (MCMC) algorithms are a well-known class of MC methods which generate a Markov chain with…
Hamiltonian Monte Carlo (HMC) algorithms which combine numerical approximation of Hamiltonian dynamics on finite intervals with stochastic refreshment and Metropolis correction are popular sampling schemes, but it is known that they may…
We introduce a powerful and flexible MCMC algorithm for stochastic simulation. The method builds on a pseudo-marginal method originally introduced in [Genetics 164 (2003) 1139--1160], showing how algorithms which are approximations to an…