Related papers: A Bayesian Theory of Change Detection in Statistic…
The Markov Decision Process (MDP) is a popular framework for sequential decision-making problems, and uncertainty quantification is an essential component of it to learn optimal decision-making strategies. In particular, a Bayesian…
We cast episodic Markov decision process (MDP) planning as Bayesian inference over policies. A policy is treated as the latent variable and is assigned an unnormalized probability of optimality that is monotone in its expected return,…
In this paper, Bayesian quickest change detection problems with sampling right constraints are considered. Specifically, there is a sequence of random variables whose probability density function will change at an unknown time. The goal is…
In multiple change-point problems, different data segments often follow different distributions, for which the changes may occur in the mean, scale or the entire distribution from one segment to another. Without the need to know the number…
This paper deals with the problem of asymptotically optimal detection of changes in regime-switching stochastic models. We need to divide the whole obtained sample of data into several sub-samples with observations belonging to different…
A novel sequential change detection problem is proposed, in which the goal is to not only detect but also accelerate the change. Specifically, it is assumed that the sequentially collected observations are responses to treatments selected…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
In this paper, we consider a modified version of the control problem in a model free Markov decision process (MDP) setting with large state and action spaces. The control problem most commonly addressed in the contemporary literature is to…
We tackle the problem of acting in an unknown finite and discrete Markov Decision Process (MDP) for which the expected shortest path from any state to any other state is bounded by a finite number $D$. An MDP consists of $S$ states and $A$…
There is a considerable amount of ongoing research on the use of Bayesian control charts for detecting a shift from a good quality distribution to a bad quality distribution in univariate and multivariate processes. It is widely claimed…
Theory and algorithms are developed for detecting changes in the distribution of statistically periodic random processes. The statistical periodicity is modeled using independent and periodically identically distributed processes, a new…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
We consider a change detection problem in which the arrival rate of a Poisson process changes suddenly at some unknown and unobservable disorder time. It is assumed that the prior distribution of the disorder time is known. The objective is…
Motivated by many application problems, we consider Markov decision processes (MDPs) with a general loss function and unknown parameters. To mitigate the epistemic uncertainty associated with unknown parameters, we take a Bayesian approach…
We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov…
Change points in real-world systems mark significant regime shifts in system dynamics, possibly triggered by exogenous or endogenous factors. These points define regimes for the time evolution of the system and are crucial for understanding…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
The process of dynamic state estimation (filtering) based on point process observations is in general intractable. Numerical sampling techniques are often practically useful, but lead to limited conceptual insight about optimal…
This paper solves a Bayes sequential impulse control problem for a diffusion, whose drift has an unobservable parameter with a change point. The partially-observed problem is reformulated into one with full observations, via a change of…
We consider a sequential Bayesian changepoint detection problem for a general stochastic model, assuming that the observed data may be dependent and non-identically distributed and the prior distribution of the change point is arbitrary,…