相关论文: Markov Chain-based Cost-Optimal Control Charts for…

Adaptive Markov chain Monte Carlo (MCMC) algorithms, which automatically tune their parameters based on past samples, have proved extremely useful in practice. The self-tuning mechanism makes them `non-Markovian', which means that their…

Markov chain Monte Carlo is a method of producing a correlated sample in order to estimate features of a target distribution via ergodic averages. A fundamental question is when should sampling stop? That is, when are the ergodic averages…

In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dataset, collected using a so-called logging policy. We develop…

A new model for controlled sensing for multihypothesis testing is proposed and studied in the sequential setting. This new model, termed {\em controlled Markovian observation} model, exhibits a more complicated memory structure in the…

The epidemic threshold of a social system is the ratio of infection and recovery rate above which a disease spreading in it becomes an epidemic. In the absence of pharmaceutical interventions (i.e. vaccines), the only way to control a given…

Design and implementation of appropriate social protection strategies is one of the main targets of the United Nation's Sustainable Development Goal (SDG) 1: No Poverty. Cash transfer (CT) programmes are considered one of the main social…

In medical imaging, it is widely recognized that image quality should be objectively evaluated based on performance in clinical tasks. To evaluate performance in signal-detection tasks, the ideal observer (IO) is optimal but also…

This paper studies the dynamic programming principle using the measurable selection method for stochastic control of continuous processes. The novelty of this work is to incorporate intermediate expectation constraints on the canonical…

This paper is concerned with a partially observed hybrid optimal control problem, where continuous dynamics and discrete events coexist and in particular, the continuous dynamics can be observed while the discrete events, described by a…

Electronic health records contain valuable information for monitoring patients' health trajectories over time. Disease progression models have been developed to understand the underlying patterns and dynamics of diseases using these data as…

We consider a continuous-time Markov chain model of SIR disease dynamics with two levels of mixing. For this so-called stochastic households model, we provide two methods for inferring the model parameters---governing within-household…

Randomized trials are considered the gold standard for making informed decisions in medicine, yet they often lack generalizability to the patient populations in clinical practice. Observational studies, on the other hand, cover a broader…

Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…

In many practical applications, decision-making processes must balance the costs of acquiring information with the benefits it provides. Traditional control systems often assume full observability, an unrealistic assumption when…

There has been an increasing use of master protocols in oncology clinical trials because of its efficiency and flexibility to accelerate cancer drug development. Depending on the study objective and design, a master protocol trial can be a…

Cardiovascular diseases are a leading cause of fatalities worldwide, often occurring suddenly with limited time for intervention. Current healthcare monitoring systems for cardiac patients rely heavily on hospitalization, which can be…

Traditional Markov Chain Monte Carlo methods suffer from low acceptance rate, slow mixing and low efficiency in high dimensions. Hamiltonian Monte Carlo resolves this issue by avoiding the random walk. Hamiltonian Monte Carlo (HMC) is a…

Orthogonal matrices play an important role in probability and statistics, particularly in high-dimensional statistical models. Parameterizing these models using orthogonal matrices facilitates dimension reduction and parameter…

In this paper, we consider the control problem with the Average-Value-at-Risk (AVaR) criteria of the possibly unbounded $L^{1}$-costs in infinite horizon on a Markov Decision Process (MDP). With a suitable state aggregation and by choosing…

This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…