Related papers: Approximation of Markov Processes by Lower Dimensi…
Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of…
We study a novel large dimensional approximate factor model with regime changes in the loadings driven by a latent first order Markov process. By exploiting the equivalent linear representation of the model, we first recover the latent…
This note describes sufficient conditions under which total-cost and average-cost Markov decision processes (MDPs) with general state and action spaces, and with weakly continuous transition probabilities, can be reduced to discounted MDPs.…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…
This paper proves that, under a monotonicity condition, the invariant probability measure of a McKean--Vlasov process can be approximated by weighted empirical measures of some processes including itself. These processes are described by…
We present a numerical method to compute expectations of functionals of a piecewise-deterministic Markov process. We discuss time dependent functionals as well as deterministic time horizon problems. Our approach is based on the…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
We consider the problem of controlling a fully specified Markov decision process (MDP), also known as the planning problem, when the state space is very large and calculating the optimal policy is intractable. Instead, we pursue the more…
In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a…
Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is…
In this paper, an alternative approximation to the innovation method is introduced for the parameter estimation of diffusion processes from partial and noisy observations. This is based on a convergent approximation to the first two…
We obtain upper bounds for the total variation distance between the distributions of two Gibbs point processes in a very general setting. Applications are provided to various well-known processes and settings from spatial statistics and…
Markov chains are an important tool for modelling and evaluating systems in computer science, economics, biology and numerous other fields. Thus, approximating Markov chains is a useful tool for decreasing the computational effort needed…
This paper presents two new approaches to decomposing and solving large Markov decision problems (MDPs), a partial decoupling method and a complete decoupling method. In these approaches, a large, stochastic decision problem is divided into…
Gradient-based algorithms are one of the methods of choice for the optimisation of Markov Decision Processes. In this article we will present a novel approximate Newton algorithm for the optimisation of such models. The algorithm has…
We propose a fast and scalable optimization method to solve chance or probabilistic constrained optimization problems governed by partial differential equations (PDEs) with high-dimensional random parameters. To address the critical…
We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method…
The generalized perturbative approach is an all purpose variant of Stein's method used to obtain rates of normal approximation. Originally developed for functions of independent random variables this method is here extended to functions of…
We propose a new method for optimistic planning in infinite-horizon discounted Markov decision processes based on the idea of adding regularization to the updates of an otherwise standard approximate value iteration procedure. This…
Herein, we analyze an efficient branching particle method for asymptotic solutions to a class of continuous-discrete filtering problems. Suppose that $t\to X_t$ is a Markov process and we wish to calculate the measure-valued process…