Related papers: Markov Chain-based Cost-Optimal Control Charts for…
We first show (1) the importance of investigating health expenditure process using the order two Markov chain model, rather than the standard order one model, which is widely used in the literature. Markov chain of order two is the minimal…
In recent years, the monitoring of compositional data using control charts has been investigated in the Statistical Process Control field. In this study, we will design a Phase II Multivariate Exponentially Weighted Moving Average (MEWMA)…
Background: All-in-one station-based health monitoring devices are implemented in elder homes in Hong Kong to support the monitoring of vital signs of the elderly. During a pilot study, it was discovered that the systolic blood pressure was…
We introduce the Lyapunov approach to optimal control problems of average risk-sensitive Markov control processes with general risk maps. Motivated by applications in particular to behavioral economics, we consider possibly non-convex risk…
We discuss various models for epidemics on networks that rely on Markov chains. Random walks on graphs are often used to predict epidemic spread and to investigate possible control actions to mitigate them. In this study, we demonstrate…
Control charts for process monitoring are widely used in practice. Most control charts require the monitored (residuals) process to be serially independent (and to satisfy specified distributional assumptions), whereas undetected dependence…
We treat the problem of risk-aware control for stochastic shortest path (SSP) on Markov decision processes (MDP). Typically, expectation is considered for SSP, which however is oblivious to the incurred risk. We present an alternative view,…
A Markov-chain model is developed for the purpose estimation of the cure rate of non-performing loans. The technique is performed collectively, on portfolios and it can be applicable in the process of calculation of credit impairment. It is…
Estimating the transition dynamics of controlled Markov chains is crucial in fields such as time series analysis, reinforcement learning, and system exploration. Traditional non-parametric density estimation methods often assume independent…
We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation…
This work focuses on optimal controls of a class of stochastic SIS epidemic models under regime switching. By assuming that a decision maker can either influence the infectivity period or isolate infected individuals, our aim is to minimize…
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…
Parametric Markov chains occur quite naturally in various applications: they can be used for a conservative analysis of probabilistic systems (no matter how the parameter is chosen, the system works to specification); they can be used to…
This paper considers a distributed decision-making approach for manufacturing task assignment and condition-based machine health maintenance. Our approach considers information sharing between the task assignment and health management…
There has been significant attention given to developing data-driven methods for tailoring patient care based on individual patient characteristics. Dynamic treatment regimes formalize this through a sequence of decision rules that map…
Estimation in the deformable template model is a big challenge in image analysis. The issue is to estimate an atlas of a population. This atlas contains a template and the corresponding geometrical variability of the observed shapes. The…
This paper deals with discrete-time Markov control processes on a general state space. A long-run risk-sensitive average cost criterion is used as a performance measure. The one-step cost function is nonnegative and possibly unbounded.…
To extend healthy life expectancy in an aging society, it is crucial to prevent various diseases at pre-disease states. Although dynamical network biomarker theory has been developed for pre-disease detection, mathematical frameworks for…
This paper studies a large number of homogeneous Markov decision processes where the transition probabilities and costs are coupled in the empirical distribution of states (also called mean-field). The state of each process is not known to…
We present a new, tractable method for solving and analyzing risk-aware control problems over finite and infinite, discounted time-horizons where the dynamics of the controlled process are described as a martingale problem. Supposing…