Related papers: Mean-Variance Type Controls Involving a Hidden Mar…
Optimal sensor scheduling with applications to networked estimation and control systems is considered. We model sensor measurement and transmission instances using jumps between states of a continuous-time Markov chain. We introduce a cost…
This paper is concerned with a stochastic linear-quadratic optimal control problem of Markovian regime switching system with model uncertainty and partial information, where the information available to the control is based on a…
In this paper, we equip the conventional discrete-time queueing network with a Markovian input process, that, in addition to the usual short-term stochastics, governs the mid- to long-term behavior of the links between the network nodes.…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
In an observed generalized semi-Markov regime, estimation of transition rate of regime switching leads towards calculation of locally risk minimizing option price. Despite the uniform convergence of estimated step function of transition…
The maximization of reach-avoid probabilities for stochastic systems is a central topic in the control literature. Yet, the available methods are either restricted to low-dimensional systems or suffer from conservative approximations. To…
This paper introduces a novel methodology for the identification of switching dynamics for switched autoregressive linear models. Switching behavior is assumed to follow a Markov model. The system's outputs are contaminated by possibly…
Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…
In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we…
We consider Markov-switching regression models, i.e. models for time series regression analyses where the functional relationship between covariates and response is subject to regime switching controlled by an unobservable Markov chain.…
We investigate nonlinear regression for nonstationary sequential data. In most real-life applications such as business domains including finance, retail, energy and economy, timeseries data exhibits nonstationarity due to the temporally…
We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model…
This work investigates the almost sure stabilization of a class of regime-switching systems based on discrete-time observations of both continuous and discrete components. It develops Shao's work [SIAM J. Control Optim., 55(2017), pp.…
We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…
We establish an abstract, effective, exponential large deviations type estimate for Markov systems satisfying a weaker form of mixing. We employ this result to derive such estimates, as well as a central limit theorem, for the skew product…
A general formalism is developed to construct a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are therefore internal to the system and not externally specified. For…
A Markov chain (MC) formalism is used to investigate the mean-square displacement (MSD) of a random walker on Newman-Watts (NW) networks. It leads to a precise analysis of the conditions for the emergence of anomalous sub- or…
We investigate an M/M/1 queue operating in two switching environments, where the switch is governed by a two-state time-homogeneous Markov chain. This model allows to describe a system that is subject to regular operating phases alternating…
We consider a control problem for a finite-state Markov system whose performance is evaluated by a coherent Markov risk measure. For each policy, the risk of a state is approximated by a function of its features, thus leading to a…
We study control variate methods for Markov chain Monte Carlo (MCMC) in the setting of deterministic sweep sampling using $K\geq 2$ transition kernels. New variance reduction results are provided for MCMC averages based on sweeps over…