Related papers: Exchangeable, Markov multi-state survival process
Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…
This paper studies continuous-time Markov decision processes under the risk-sensitive average cost criterion. The state space is a finite set, the action space is a Borel space, the cost and transition rates are bounded, and the…
In this paper, we reveal a general relationship between model simplification and irreversibility based on the model of continuous-time Markov chains with time-scale separation. According to the topological structure of the fast process, we…
In this paper we study the long term evolution of a continuous time Markov chain formed by two interacting birth-and-death processes. The interaction between the processes is modelled by transition rates which are functions with suitable…
The analysis of multiple time-to-event outcomes in a randomised controlled clinical trial can be accomplished with exisiting methods. However, depending on the characteristics of the disease under investigation and the circumstances in…
In modern life insurance, Markov processes in continuous time on a finite or at least countable state space have been over the years an important tool for the modelling of the states of an insured. Motivated by applications in disability…
Let $(X_t)_{t \geq 0}$ be a continuous time Markov process on some metric space $M,$ leaving invariant a closed subset $M_0 \subset M,$ called the {\em extinction set}. We give general conditions ensuring either "Stochastic persistence"…
Any exchangeable Markov processes on $[k]^{\mathbb{N}}$ with cadlag sample paths projects to a Markov process on the simplex whose sample paths are cadlag and of locally bounded variation. Furthermore, any such process has a de Finetti-type…
The initial transient phase of an emerging epidemic is of critical importance for data-driven model building, model-based prediction of the epidemic trend, and articulation of control/prevention strategies. In principle, quantitative models…
For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…
This paper investigates MDPs with intermittent state information. We consider a scenario where the controller perceives the state information of the process via an unreliable communication channel. The transmissions of state information…
We consider a random walk with catastrophes which was introduced to model population biology. It is known that this Markov chain gets eventually absorbed at $0$ for all parameter values. Recently, it has been shown that this chain exhibits…
Species sampling processes have long served as the fundamental framework for modeling random discrete distributions and exchangeable sequences. However, data arising from distinct but related sources require a broader notion of…
Markov chains have long been used for generating random variates from spatial point processes. Broadly speaking, these chains fall into two categories: Metropolis-Hastings type chains running in discrete time and spatial birth-death chains…
We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities,…
Stochastic processes can model many emerging phenomena on networks, like the spread of computer viruses, rumors, or infectious diseases. Understanding the dynamics of such stochastic spreading processes is therefore of fundamental interest.…
We describe a Markov latent state space (MLSS) model, where the latent state distribution is a decaying mixture over multiple past states. We present a simple sampling algorithm that allows to approximate such high-order MLSS with fixed…
In this paper, we consider statistical estimation of time-inhomogeneous aggregate Markov models. Unaggregated models, which corresponds to Markov chains, are commonly used in multi-state life insurance to model the biometric states of an…
We consider the modeling of data generated by a latent continuous-time Markov jump process with a state space of finite but unknown dimensions. Typically in such models, the number of states has to be pre-specified, and Bayesian inference…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…