Related papers: Controlled Martingale Problems And Their Markov Mi…
We study Markov decision processes with Polish state and action spaces. The action space is state dependent and is not necessarily compact. We first establish the existence of an optimal ergodic occupation measure using only a near-monotone…
In many operations management problems, we need to make decisions sequentially to minimize the cost while satisfying certain constraints. One modeling approach to study such problems is constrained Markov decision process (CMDP). When…
In this paper we survey the almost sure central limit theorem and its functional form (quenched) for stationary and ergodic processes. For additive functionals of a stationary and ergodic Markov chain these theorems are known under the…
We discretize a risk-neutral optimal control problem governed by a linear elliptic partial differential equation with random inputs using a Monte Carlo sample-based approximation and a finite element discretization, yielding finite…
This paper considers an infinite-horizon Markov decision process (MDP) that allows for general non-exponential discount functions, in both discrete and continuous time. Due to the inherent time inconsistency, we look for a randomized…
The martingale expansion provides a refined approximation to the marginal distributions of martingales beyond the normal approximation implied by the martingale central limit theorem. We develop a martingale expansion framework specifically…
We show that, for first-order systems of conservation laws with a strictly convex entropy,in particular for the very simple so-called "inviscid" Burgers equation,it is possible to address the Cauchy problem by a suitable convex…
In this paper, we prove the existence of minimizers of a class of multi-constrained variational problems. We consider systems involving a nonlinearity that does not satisfy compactness, monotonicity, neither symmetry properties. Our…
Entropic Optimal Transport (EOT), also referred to as the Schr\"odinger problem, seeks to find a random processes with prescribed initial/final marginals and with minimal relative entropy with respect to a reference measure. The relative…
Recently, we introduced a solution to the quantum marginal problem relevant to two-dimensional quantum many-body systems [I. H. Kim, Phys. Rev. X, 11, 021039]. One of the conditions was that the marginals are internally translationally…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…
This article deals with the numerical approximation of Markovian backward stochastic differential equations (BSDEs) with generators of quadratic growth with respect to $z$ and bounded terminal conditions. We first study a slight…
We consider a semi-Lagrangian scheme for solving the minimum time problem, with a given target, and the associated eikonal type equation. We first use a discrete time deterministic optimal control problem interpretation of the time…
We present a hierarchy of tractable relaxations to obtain lower bounds on the minimum value of a polynomial over a constraint set defined by polynomial equations. In contrast to previous convex relaxation techniques for this problem, our…
It is well-known that well-posedness of a martingale problem in the class of continuous (or r.c.l.l.) solutions enables one to construct the associated transition probability functions. We extend this result to the case when the martingale…
Markov chain Monte Carlo sampling methods often suffer from long correlation times. Consequently, these methods must be run for many steps to generate an independent sample. In this paper a method is proposed to overcome this difficulty.…
Let $G$ be a semimartingale, and $S$ its Snell envelope. Under the assumption that $G\in\mathcal{H}^1$, we show that the finite-variation part of $S$ is absolutely continuous with respect to the decreasing part of the finite-variation part…
We focus on a class of BSDEs driven by a cadlag martingale and corresponding Markov type BSDE which arise when the randomness of the driver appears through a Markov process. To those BSDEs we associate a deterministic problem which, when…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…
We study the problem of approximation of solutions of the Skorokhod problem and reflecting stochastic differential equations (SDEs) with jumps by sequences of solutions of equations with penalization terms. Applications to discrete…