Related papers: A Bayesian Theory of Change Detection in Statistic…
Optimal control in non-stationary Markov decision processes (MDP) is a challenging problem. The aim in such a control problem is to maximize the long-term discounted reward when the transition dynamics or the reward function can change over…
A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. Algorithms are proposed to detect changes in such i.p.i.d.…
Models of many real-life applications, such as queuing models of communication networks or computing systems, have a countably infinite state-space. Algorithmic and learning procedures that have been developed to produce optimal policies…
We show that the optimal decision policy for several types of Bayesian sequential detection problems has a threshold switching curve structure on the space of posterior distributions. This is established by using lattice programming and…
We study an optimal process control problem with multiple assignable causes. The process is initially in-control but is subject to random transition to one of multiple out-of-control states due to assignable causes. The objective is to find…
In 1960s Shiryaev developed Bayesian theory of change detection in independent and identically distributed (i.i.d.) sequences. In Shiryaev's classical setting the goal is to minimize an average detection delay under the constraint imposed…
Sequential change diagnosis is the joint problem of detection and identification of a sudden and unobservable change in the distribution of a random sequence. In this problem, the common probability law of a sequence of i.i.d. random…
Stochastic optimal control with unknown randomness distributions has been studied for a long time, encompassing robust control, distributionally robust control, and adaptive control. We propose a new episodic Bayesian approach that…
We consider a change-point detection problem for a simple class of Piecewise Deterministic Markov Processes (PDMPs). A continuous-time PDMP is observed in discrete time and through noise, and the aim is to propose a numerical method to…
Change point detection becomes more and more important as datasets increase in size, where unsupervised detection algorithms can help users process data. To detect change points, a number of unsupervised algorithms have been developed which…
Addressing uncertainty is critical for autonomous systems to robustly adapt to the real world. We formulate the problem of model uncertainty as a continuous Bayes-Adaptive Markov Decision Process (BAMDP), where an agent maintains a…
In the 1960s, Shiryaev developed a Bayesian theory of change-point detection in the i.i.d. case, which was generalized in the beginning of the 2000s by Tartakovsky and Veeravalli for general stochastic models assuming a certain stability of…
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…
The problem of sequentially detecting an abrupt change in a sequence of independent and identically distributed (IID) random variables is addressed. Whereas previous approaches assume a known probability density function (PDF) at the start…
We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…
Adaptive control problems are notoriously difficult to solve even in the presence of plant-specific controllers. One way to by-pass the intractable computation of the optimal policy is to restate the adaptive control as the minimization of…
Suppose that local characteristics of several independent compound Poisson and Wiener processes change suddenly and simultaneously at some unobservable disorder time. The problem is to detect the disorder time as quickly as possible after…
We consider the control of a Markov decision process (MDP) that undergoes an abrupt change in its transition kernel (mode). We formulate the problem of minimizing regret under control-switching based on mode change detection, compared to a…
We study Bayesian optimal control of a general class of smoothly parameterized Markov decision problems. Since computing the optimal control is computationally expensive, we design an algorithm that trades off performance for computational…
In this paper, the problem of quickly detecting an abrupt change on a stochastic process under Bayesian framework is considered. Different from the classic Bayesian quickest change-point detection problem, this paper considers the case…