Related papers: Partially observed optimal stopping problem for di…
Herein, we analyze an efficient branching particle method for asymptotic solutions to a class of continuous-discrete filtering problems. Suppose that $t\to X_t$ is a Markov process and we wish to calculate the measure-valued process…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
Optimization problems with $L^1$-control cost functional subject to an elliptic partial differential equation (PDE) are considered. However, different from the finite dimensional $l^1$-regularization optimization, the resulting discretized…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
We derive a reduced-order state estimator for discrete-time infinite dimensional linear systems with finite dimensional Gaussian input and output noise. This state estimator is the optimal one-step estimate that takes values in a fixed…
In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…
We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point…
Discrete time stochastic optimal control problems and Markov decision processes (MDPs), respectively, serve as fundamental models for problems that involve sequential decision making under uncertainty and as such constitute the theoretical…
In this paper, an efficient parallel splitting method is proposed for the optimal control problem with parabolic equation constraints. The linear finite element is used to approximate the state variable and the control variable in spatial…
In a classical optimal stopping problem the aim is to maximize the expected value of a functional of a diffusion evaluated at a stopping time. This note considers optimal stopping problems beyond this paradigm. We study problems in which…
Numerical analysis for the stochastic Stokes equations is still challenging even though it has been well done for the corresponding deterministic equations. In particular, the pre-existing error estimates of finite element methods for the…
We study stochastic optimal control problems for (possibly degenerate) McKean-Vlasov controlled diffusions and obtain discrete-time as well as finite interacting particle approximations. (i) Under mild assumptions, we first prove the…
This paper develops and analyzes an optimal-order semi-discrete scheme and its fully discrete finite element approximation for nonlinear stochastic elastic wave equations with multiplicative noise. A non-standard time-stepping scheme is…
In this paper, we consider the optimal stopping problem on semi-Markov processes (SMPs) with finite horizon, and aim to establish the existence and computation of optimal stopping times. To achieve the goal, we first develop the main…
The process of dynamic state estimation (filtering) based on point process observations is in general intractable. Numerical sampling techniques are often practically useful, but lead to limited conceptual insight about optimal…
We study decision timing problems on finite horizon with Poissonian information arrivals. In our model, a decision maker wishes to optimally time her action in order to maximize her expected reward. The reward depends on an unobservable…
For a discrete time Markov chain and in line with Strotz' consistent planning we develop a framework for problems of optimal stopping that are time-inconsistent due to the consideration of a non-linear function of an expected reward. We…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
This work addresses stochastic optimal control problems where the unknown state evolves in continuous time while partial, noisy, and possibly controllable measurements are only available in discrete time. We develop a framework for…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…