Related papers: Mean-Variance Type Controls Involving a Hidden Mar…
In this paper, we propose a chance constrained stochastic model predictive control scheme for reference tracking of distributed linear time-invariant systems with additive stochastic uncertainty. The chance constraints are reformulated…
We consider a continuous-time financial market with an asset whose price is modeled by a linear stochastic differential equation with drift and volatility switching driven by a uniformly ergodic jump Markov process with a countable state…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
We study a novel large dimensional approximate factor model with regime changes in the loadings driven by a latent first order Markov process. By exploiting the equivalent linear representation of the model, we first recover the latent…
Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…
We study a variance reduction strategy based on control variables for simulating the averaged macroscopic behavior of a stochastic slow-fast system. We assume that this averaged behavior can be written in terms of a few slow degrees of…
The paper deals with a mathematical model of a surveillance system based on a net of sensors. The signals acquired by each node of the net are Markovian process, have two different transition probabilities, which depends on the presence or…
Many real-world phenomena can be modelled as dynamical processes on networks, a prominent example being the spread of infectious diseases such as COVID-19. Mean-field approximations are a widely used tool to analyse such dynamical processes…
This article presents a novel class of control policies for networked control of Lyapunov-stable linear systems with bounded inputs. The control channel is assumed to have i.i.d. Bernoulli packet dropouts and the system is assumed to be…
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results…
We demonstrate the use of a variational method to determine a quantitative lower bound on the rate of convergence of Markov Chain Monte Carlo (MCMC) algorithms as a function of the target density and proposal density. The bound relies on…
Event-triggered approach to networked control systems is used to reduce the workload of the communication network. For the static output-feedback continuous event-trigger may generate an infinite number of sampling instants in finite time…
In this work we provide a computationally tractable procedure for designing affine control policies, applied to constrained, discrete-time, partially observable, linear systems subject to set bounded disturbances, stochastic noise and…
We consider the discrete-time filtering problem in scenarios where the observation noise is degenerate or low. More precisely, one is given access to a discrete time observation sequence which at any time $k$ depends only on the state of an…
The number of observable degrees of freedom is typically limited in experiments. Here, we consider discrete Markov networks in which an observer has access to a few visible transitions and the waiting times between these transitions.…
One of the most fundamental problems in Markov decision processes is analysis and control synthesis for safety and reachability specifications. We consider the stochastic reach-avoid problem, in which the objective is to synthesize a…
This chapter presents an introduction to Markovian modeling for the analysis of sequence data. Contrary to the deterministic approach seen in the previous sequence analysis chapters, Markovian models are probabilistic models, focusing on…
A hidden Markov process is a well known concept in information theory and is used for a vast range of applications such as speech recognition and error correction. We bridge between two disciplines, experimental physics and advanced…
Continuous-time Bayesian networks is a natural structured representation language for multicomponent stochastic processes that evolve continuously over time. Despite the compact representation, inference in such models is intractable even…