Related papers: Optimal stopping for partially observed piecewise-…
In this paper we consider a parabolic optimal control problem with a Dirac type control with moving point source in two space dimensions. We discretize the problem with piecewise constant functions in time and continuous piecewise linear…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is…
We consider a broad class of dynamic programming (DP) problems that involve a partially linear structure and some positivity properties in their system equation and cost function. We address deterministic and stochastic problems, possibly…
Particle filtering is a powerful tool for target tracking. When the budget for observations is restricted, it is necessary to reduce the measurements to a limited amount of samples carefully selected. A discrete stochastic nonlinear…
In this paper is described the general aspect of a numerical method for piecewise determin-istic Markov processes with boundary. Under very natural hypotheses, a crucial result about uniqueness of solution of a generalized Kolmogorov…
We study optimal control problems in infinite horizon when the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (inspired by traffic models). We adapt the results in [H. M.…
This paper is devoted to filtering, smoothing, and prediction of polynomial processes that are partially observed. These problems are known to allow for an explicit solution in the simpler case of linear Gaussian state space models. The key…
The paper addresses two variants of the stochastic shortest path problem ('optimize the accumulated weight until reaching a goal state') in Markov decision processes (MDPs) with integer weights. The first variant optimizes partial expected…
We consider a semi-Lagrangian scheme for solving the minimum time problem, with a given target, and the associated eikonal type equation. We first use a discrete time deterministic optimal control problem interpretation of the time…
In this paper, we are interested in optimal decisions in a partially observable Markov universe. Our viewpoint departs from the dynamic programming viewpoint: we are directly approximating an optimal strategic tree depending on the…
Partially observable Markov decision processes (POMDPs) have recently become popular among many AI researchers because they serve as a natural model for planning under uncertainty. Value iteration is a well-known algorithm for finding…
We consider an investor faced with the utility maximization problem in which the risky asset price process has pure-jump dynamics affected by an unobservable continuous-time finite-state Markov chain, the intensity of which can also be…
We study the problem of synthesizing a controller that maximizes the entropy of a partially observable Markov decision process (POMDP) subject to a constraint on the expected total reward. Such a controller minimizes the predictability of a…
In this paper we formulate and study an optimal switching problem under partial information. In our model the agent/manager/investor attempts to maximize the expected reward by switching between different states/investments. However, he is…
Discrete time control systems whose dynamics and observations are described by stochastic equations are common in engineering, operations research, health care, and economics. For example, stochastic filtering problems are usually defined…
This paper deals with the question of how to most effectively conduct experiments in Partially Observed Markov Decision Processes so as to provide data that is most informative about a parameter of interest. Methods from Markov decision…
We study the optimal stopping problem of McKean-Vlasov diffusions when the criterion is a function of the law of the stopped process. A remarkable new feature in this setting is that the stopping time also impacts the dynamics of the…
In this Note we study optimal stopping problems for strong Markov processes and affine functions. We give a justification of the Snell envelope form using standard results of optimal stopping. We also justify the convexity of the value…
We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albeit with a number of…