Related papers: A Bayesian Theory of Change Detection in Statistic…
This paper addresses the issue of detecting change-points in multivariate time series. The proposed approach differs from existing counterparts by making only weak assumptions on both the change-points structure across series, and the…
A new model for controlled sensing for multihypothesis testing is proposed and studied in the sequential setting. This new model, termed {\em controlled Markovian observation} model, exhibits a more complicated memory structure in the…
Model Predictive Control (MPC) is a powerful framework for constrained control, but its performance and safety can be severely degraded when the prediction model is learned online and thus remains uncertain. In this work, we develop a…
Policy Iteration (PI) is a widely used family of algorithms to compute optimal policies for Markov Decision Problems (MDPs). We derive upper bounds on the running time of PI on Deterministic MDPs (DMDPs): the class of MDPs in which every…
In crowd labeling, a large amount of unlabeled data instances are outsourced to a crowd of workers. Workers will be paid for each label they provide, but the labeling requester usually has only a limited amount of the budget. Since data…
In the classical quickest detection problem, one must detect as quickly as possible when a Brownian motion without drift "changes" into a Brownian motion with positive drift. The change occurs at an unknown "disorder" time with exponential…
The Bayesian inversion method demonstrates significant potential for solving inverse problems, enabling both point estimation and uncertainty quantification (UQ). However, Bayesian maximum a posteriori (MAP) estimation may become unstable…
Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics…
Discrete time control systems whose dynamics and observations are described by stochastic equations are common in engineering, operations research, health care, and economics. For example, stochastic filtering problems are usually defined…
We provide a bird's eye view onto the area of sequential change-point detection. We focus on the discrete-time case with known pre- and post-change data distributions and offer a summary of the forefront asymptotic results established in…
Economic Model Predictive Control (MPC) dissipativity theory is central to discussing the stability of policies resulting from minimizing economic stage costs. In its current form, the dissipativity theory for economic MPC applies to…
Changepoint models typically assume the data within each segment are independent and identically distributed conditional on some parameters which change across segments. This construction may be inadequate when data are subject to local…
In this paper we address the class of Sequential Decision Making (SDM) problems that are characterized by time-varying parameters. These parameter dynamics are either pre-specified or manipulable. At any given time instant the decision…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
Motivated from Bertsekas' recent study on policy iteration (PI) for solving the problems of infinite-horizon discounted Markov decision processes (MDPs) in an on-line setting, we develop an off-line PI integrated with a multi-policy…
Since the work of Page in the 1950s, the problem of detecting an abrupt change in the distribution of stochastic processes has received a great deal of attention. In particular, a deep connection has been established between Lorden's…
In this paper, we study a mean-variance optimization problem in an infinite horizon discrete time discounted Markov decision process (MDP). The objective is to minimize the variance of system rewards with the constraint of mean performance.…
Although the applications of Non-Homogeneous Poisson Processes to model and study the threshold overshoots of interest in different time series of measurements have proven to provide good results, they needed to be complemented with an…
In modern business modeling and analytics, data monitoring plays a critical role. Nowadays, sophisticated models often rely on hundreds or even thousands of input variables. Over time, structural changes such as abrupt level shifts or trend…
This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is…