Related papers: Sequential tracking of an unobservable two-state M…
We construct a Bayesian sequential test of two simple hypotheses about the value of the unobservable drift coefficient of a Brownian motion, with a possibility to change the initial decision at subsequent moments of time for some penalty.…
In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…
Consider the motion of a Brownian particle in two or more dimensions, whose coordinate processes are standard Brownian motions with zero drift initially, and then at some random/unobservable time, one of the coordinate processes gets a…
We study the problem of optimally managing an inventory with unknown demand trend. Our formulation leads to a stochastic control problem under partial observation, in which a Brownian motion with non-observable drift can be singularly…
This paper is concerned with a characterization of the observability for a continuous-time hidden Markov model where the state evolves as a general continuous-time Markov process and the observation process is modeled as nonlinear function…
We study a classical Bayesian statistics problem of sequentially testing the sign of the drift of an arithmetic Brownian motion with the $0$-$1$ loss function and a constant cost of observation per unit of time for general prior…
We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…
This paper solves a Bayes sequential impulse control problem for a diffusion, whose drift has an unobservable parameter with a change point. The partially-observed problem is reformulated into one with full observations, via a change of…
The multiple disorder problem seeks to determine a sequence of stopping times which are as close as possible to the unknown times of disorders at which the observation process changes its probability characteristics. We derive closed form…
In the classical quickest detection problem, one must detect as quickly as possible when a Brownian motion without drift "changes" into a Brownian motion with positive drift. The change occurs at an unknown "disorder" time with exponential…
We study finite horizon optimal switching problems for hidden Markov chain models under partially observable Poisson processes. The controller possesses a finite range of strategies and attempts to track the state of the unobserved state…
We consider an infinite horizon optimal control problem for a pure jump Markov process $X$, taking values in a complete and separable metric space $I$, with noise-free partial observation. The observation process is defined as $Y_t =…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…
The problem of detecting a change in the drift of a Brownian motion is considered. The change point is assumed to have a modified exponential prior distribution with unknown parameters. A worst-case analysis with respect to these parameters…
In this paper we study the stochastic control problem of partially observed (multi-dimensional) stochastic system driven by both Brownian motions and fractional Brownian motions. In the absence of the powerful tool of Girsanov…
We formulate and solve a variant of the quickest detection problem which features false negatives. A standard Brownian motion acquires a drift at an independent exponential random time which is not directly observable. Based on the…
We study a controlled version of the Bayesian sequential testing problem for the drift of a Wiener process, in which the observer exercises discretion over the signal intensity. This control incurs a running cost that reflects the resource…
In this short note we will provide a sufficient and necessary condition to have uniqueness of the location of the maximum of a stochastic process over an interval. The result will also express the mean value of the location in terms of the…
In this paper we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time…