Related papers: Mean-Variance Type Controls Involving a Hidden Mar…
In this paper, we study a mean-variance optimization problem in an infinite horizon discrete time discounted Markov decision process (MDP). The objective is to minimize the variance of system rewards with the constraint of mean performance.…
This article is concerned with stability analysis and stabilization of randomly switched systems under a class of switching signals. The switching signal is modeled as a jump stochastic (not necessarily Markovian) process independent of the…
In the regime switching extension of Black-Scholes-Merton model of asset price dynamics, one assumes that the volatility coefficient evolves as a hidden pure jump process. Under the assumption of Markov regime switching, we have considered…
In this paper, we consider the stability analysis of large-scale distributed networked control systems with random communication delays between linearly interconnected subsystems. The stability analysis is performed in the Markov jump…
Mean-field models are a popular tool in a variety of fields. They provide an understanding of the impact of interactions among a large number of particles or people or other "self-interested agents", and are an increasingly popular tool in…
In this paper, selection of an active sensor subset for tracking a discrete time, finite state Markov chain having an unknown transition probability matrix (TPM) is considered. A total of N sensors are available for making observations of…
This paper studies a large number of homogeneous Markov decision processes where the transition probabilities and costs are coupled in the empirical distribution of states (also called mean-field). The state of each process is not known to…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
In this paper we present elementary computations for some Markov modulated counting processes, also called counting processes with regime switching. Regime switching has become an increasingly popular concept in many branches of science. In…
This paper introduces a novel model-based clustering approach for clustering time series which present changes in regime. It consists of a mixture of polynomial regressions governed by hidden Markov chains. The underlying hidden process for…
Markov chain Monte Carlo (MCMC) is the engine of modern Bayesian statistics, being used to approximate the posterior and derived quantities of interest. Despite this, the issue of how the output from a Markov chain is post-processed and…
Herein, the Hidden Markov Model is expanded to allow for Markov chain observations. In particular, the observations are assumed to be a Markov chain whose one step transition probabilities depend upon the hidden Markov chain. An…
We aim at studying approximate null-controllability properties of a particular class of piecewise linear Markov processes (Markovian switch systems). The criteria are given in terms of algebraic invariance and are easily computable. We…
This work focuses on optimal harvesting-renewing for a stochastic population. A mixed regular-singular control formulation with a state constraint and regime-switching is introduced. The decision-makers either harvest or renew with finite…
Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…
Estimating the transition dynamics of controlled Markov chains is crucial in fields such as time series analysis, reinforcement learning, and system exploration. Traditional non-parametric density estimation methods often assume independent…
In this paper, we introduce a novel approach to solve the (mean-covariance) steering problem for a fairly general class of linear continuous-time stochastic systems subject to input delays. Specifically, we aim at steering delayed linear…
A regime-switching multivariate time series model which is closed under margins is built. The model imposes a restriction on all lower-dimensional sub-processes to follow a regime-switching process sharing the same latent regime sequence…
This paper investigates an important class of information-flow security property called opacity for stochastic control systems. Opacity captures whether a system's secret behavior (a subset of the system's behavior that is considered to be…
Motivated by reduction of computational complexity, this work develops sign-error adaptive filtering algorithms for estimating time-varying system parameters. Different from the previous work on sign-error algorithms, the parameters are…