Related papers: Numerical method for optimal stopping of piecewise…
This paper combines data-driven and model-driven methods for real-time misinformation detection. Our algorithm, named QuickStop, is an optimal stopping algorithm based on a probabilistic information spreading model obtained from labeled…
Value iteration is a well-known method of solving Markov Decision Processes (MDPs) that is simple to implement and boasts strong theoretical convergence guarantees. However, the computational cost of value iteration quickly becomes…
We give polynomial time algorithms for quantitative (and qualitative) reachability analysis for Branching Markov Decision Processes (BMDPs). Specifically, given a BMDP, and given an initial population, where the objective of the controller…
This paper considers a multiple stopping time problem for a Markov chain observed in noise, where a decision maker chooses at most L stopping times to maximize a cumulative objective. We formulate the problem as a Partially Observed Markov…
We consider a control constrained parabolic optimal control problem and use variational discretization for its time semi-discretization. The state equation is treated with a Petrov-Galerkin scheme using a piecewise constant Ansatz for the…
We investigate a singular-optimal stopping stochastic control problem driven by self-exciting dynamics governed by a Hawkes process. In the continuous-time setting, we show that the optimization problem reduces to solving a variational…
One often encounters the curse of dimensionality in the application of dynamic programming to determine optimal policies for controlled Markov chains. In this paper, we provide a method to construct sub-optimal policies along with a bound…
We study the numerical solution of nonlinear partially observed optimal stopping problems. The system state is taken to be a multi-dimensional diffusion and drives the drift of the observation process, which is another multi-dimensional…
This article introduces a numerical algorithm that serves as a preliminary step toward solving continuous-time model predictive control (MPC) problems directly without explicit time-discretization. The chief ingredients of the underlying…
In this paper, we study a mean-variance optimization problem in an infinite horizon discrete time discounted Markov decision process (MDP). The objective is to minimize the variance of system rewards with the constraint of mean performance.…
This paper shows that the optimal policy and value functions of a Markov Decision Process (MDP), either discounted or not, can be captured by a finite-horizon undiscounted Optimal Control Problem (OCP), even if based on an inexact model.…
In this article we propose a Weighted Stochastic Mesh (WSM) Algorithm for approximating the value of a discrete and continuous time optimal stopping problem. We prove that in the discrete case the WSM algorithm leads to semi-tractability of…
In this paper, we solve explicitly the optimal stopping problem with random discounting and an additive functional as cost of observations for a regular linear diffusion. We also extend the results to the class of one-sided regular Feller…
In this paper we consider a class of piecewise-deterministic Markov processes (PDMPs) modeling the quantity of a given food contaminant in the body. On the one hand, the amount of contaminant increases with random food intakes and, on the…
Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…
We use the geometry of suitably generalised potentials to solve risk-sensitive Markovian optimal stopping problems. As in the linear case due to Dynkin and Yushkievich (1967), the value function is the pointwise infimum of those functions…
We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point…
We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov…
A general result on the method of randomized stopping is proved. It is applied to optimal stopping of controlled diffusion processes with unbounded coefficients to reduce it to an optimal control problem without stopping. This is motivated…
We investigate an optimal stopping problem for the expected value of a discounted payoff on a regime-switching geometric Brownian motion under two constraints on the possible stopping times: only at exogenous random times and only during a…