Related papers: Sequential Tracking of a Hidden Markov Chain Using…

This paper proposes a new randomized strategy for adaptive MCMC using Bayesian optimization. This approach applies to non-differentiable objective functions and trades off exploration and exploitation to reduce the number of potentially…

We investigate the problem of monitoring partially observable systems with nondeterministic and probabilistic dynamics. In such systems, every state may be associated with a risk, e.g., the probability of an imminent crash. During runtime,…

Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics…

Reversibility is a key property of Markov chains, central to algorithms such as Metropolis-Hastings and other MCMC methods. Yet many applications yield non-reversible chains, motivating the problem of approximating them by reversible ones…

Safety-critical cyber-physical systems require control strategies whose worst-case performance is robust against adversarial disturbances and modeling uncertainties. In this paper, we present a framework for approximate control and learning…

This paper introduces a new approach of treating platoon systems using mean-variance control formulation. The underlying system is a controlled switching diffusion in which the random switching process is a continuous-time Markov chain.…

Motivated by applications in movement ecology, in this paper I propose a new class of integrated continuous-time hidden Markov models in which each observation depends on the underlying state of the process over the whole interval since the…

We develop a predictive-first optimisation framework for streaming hidden Markov models. Unlike classical approaches that prioritise full posterior recovery under a fully specified generative model, we assume access to regime-specific…

Markov chains are simple yet powerful mathematical structures to model temporally dependent processes. They generally assume stationary data, i.e., fixed transition probabilities between observations/states. However, live, real-world…

This paper solves a Bayes sequential impulse control problem for a diffusion, whose drift has an unobservable parameter with a change point. The partially-observed problem is reformulated into one with full observations, via a change of…

A Markov process is registered. At random moment $\theta$ the distribution of observed sequence changes. Using probability maximizing approach the optimal stopping rule for detecting the change is identified. Some explicit solution is…

We consider a control problem for a finite-state Markov system whose performance is evaluated by a coherent Markov risk measure. For each policy, the risk of a state is approximated by a function of its features, thus leading to a…

In this article, we are interested in planning problems where the agent is aware of the presence of an observer, and where this observer is in a partial observability situation. The agent has to choose its strategy so as to optimize the…

In this paper we formulate and study an optimal switching problem under partial information. In our model the agent/manager/investor attempts to maximize the expected reward by switching between different states/investments. However, he is…

We study the problem of optimal portfolio selection in an illiquid market with discrete order flow. In this market, bids and offers are not available at any time but trading occurs more frequently near a terminal horizon. The investor can…

Output-Feedback Stochastic Model Predictive Control based on Stochastic Optimal Control for nonlinear systems is computationally intractable because of the need to solve a Finite Horizon Stochastic Optimal Control Problem. However, solving…

A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…

We consider an optimal switching problem where the terminal reward depends on the entire control trajectory. We show existence of an optimal control by applying a probabilistic technique based on the concept of Snell envelopes. We then…

We consider both discrete and continuous "uncertain horizon" deterministic control processes, for which the termination time is a random variable. We examine the dynamic programming equations for the value function of such processes,…

Process monitoring and control requires detection of structural changes in a data stream in real time. This article introduces an efficient sequential Monte Carlo algorithm designed for learning unknown changepoints in continuous time. The…