Related papers: Sequential Tracking of a Hidden Markov Chain Using…
The process of dynamic state estimation (filtering) based on point process observations is in general intractable. Numerical sampling techniques are often practically useful, but lead to limited conceptual insight about optimal…
The problem of appropriately matching items subject to compatibility constraints arises in a number of important applications. While most of the literature on matching theory focuses on a static setting with a fixed number of items, several…
Active state tracking is needed in object classification, target tracking, medical diagnosis and estimation of sparse signals among other various applications. Herein, active state tracking of a discrete-time, finite-state Markov chain is…
This paper concerns discrete-time infinite-horizon stochastic control systems with Borel state and action spaces and universally measurable policies. We study optimization problems on strategic measures induced by the policies in these…
In this paper we study the valuation problem of an insurance company by maximizing the expected discounted future dividend payments in a model with partial information that allows for a changing economic environment. The surplus process is…
In this paper we study the Poisson Hypothesis, which is a device to analyze approximately the behavior of large queueing networks. We prove it in some simple limiting cases. We show in particular that the corresponding dynamical system,…
The statistics of the diffusive motion of particles often serve as an experimental proxy for their interaction with the environment. However, inferring the physical properties from the observed trajectories is challenging. Inspired by a…
We model the stock price dynamics through a semi-Markov process obtained using a Poisson random measure. We establish the existence and uniqueness of the classical solution of a non-homogeneous terminal value problem and we show that the…
This work focuses on a self-exciting point process defined by a Hawkes-like intensity and a switching mechanism based on a hidden Markov chain. Previous works in such a setting assume constant intensities between consecutive events. We…
A decision maker records measurements of a finite-state Markov chain corrupted by noise. The goal is to decide when the Markov chain hits a specific target state. The decision maker can choose from a finite set of sampling intervals to pick…
We consider a collection of statistically identical two-state continuous time Markov chains (channels). A controller continuously selects a channel with the view of maximizing infinite horizon average reward. A switching cost is paid upon…
The problem of state tracking with active observation control is considered for a system modeled by a discrete-time, finite-state Markov chain observed through conditionally Gaussian measurement vectors. The measurement model statistics are…
Qualitative opacity of a secret is a security property, which means that a system trajectory satisfying the secret is observation-equivalent to a trajectory violating the secret. In this paper, we study how to synthesize a control policy…
This article surveys recent advancements of strategy designs for persistent robotic surveillance tasks with the focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way where…
We consider the analysis of sets of categorical sequences consisting of piecewise homogeneous Markov segments. The sequences are assumed to be governed by a common underlying process with segments occurring in the same order for each…
We consider an infinite horizon optimal control problem for a pure jump Markov process $X$, taking values in a complete and separable metric space $I$, with noise-free partial observation. The observation process is defined as $Y_t =…
Automatically recognizing the e-learning activities is an important task for improving the online learning process. Probabilistic graphical models such as hidden Markov models and conditional random fields have been successfully used in…
Starting from the Avellaneda-Stoikov framework, we consider a market maker who wants to optimally set bid/ask quotes over a finite time horizon, to maximize her expected utility. The intensities of the orders she receives depend not only on…
In this paper, we are interested in optimal decisions in a partially observable Markov universe. Our viewpoint departs from the dynamic programming viewpoint: we are directly approximating an optimal strategic tree depending on the…
This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of epsilon-optimal strategies. The construction of such strategies is explicit and only…