Related papers: Some notes on a semi-Markov matrix occurring in th…

Markov-modulated L\'evy processes lead to matrix integral equations of the kind $ A_0 + A_1X+A_2 X^2+A_3(X)=0$ where $A_0$, $A_1$, $A_2$ are given matrix coefficients, while $A_3(X)$ is a nonlinear function, expressed in terms of integrals…

The goal of a traditional Markov decision process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly infinite). In many applications, however, a decision maker may be interested in optimizing a specific…

The embedding problem of Markov matrices in Markov semigroups is a classic problem that regained a lot of impetus and activities through recent needs in phylogeny and population genetics. Here, we give an account for dimensions $d\leqslant…

This note presents conjectures on polynomial/algebraic/sub-exponential convergence of transition probabilities for $\lambda$-null recurrent and $\lambda$-transient Markov chains in continuous time. The only known positive examples are in…

We investigate an M/M/1 queue operating in two switching environments, where the switch is governed by a two-state time-homogeneous Markov chain. This model allows to describe a system that is subject to regular operating phases alternating…

Markov Population Model is a commonly used framework to describe stochastic systems. Their exact analysis is unfeasible in most cases because of the state space explosion. Approximations are usually sought, often with the goal of reducing…

Counterfactuals are widely used in AI to explain how minimal changes to a model's input can lead to a different output. However, established methods for computing counterfactuals typically focus on one-step decision-making, and are not…

We consider parameter estimation in a regression model corresponding to an iid sequence of censored observations of a finite state modulated renewal process. The model assumes a similar form as in Cox regression except that the baseline…

A fluid queue is a stochastic process which moves linearly with a rate that is determined by the state of a continuous-time Markov chain (CTMC). In this paper we construct an approximation to a fluid queue using a quasi birth-and-death…

Branching processes are a class of continuous-time Markov chains (CTMCs) with ubiquitous applications. A general difficulty in statistical inference under partially observed CTMC models arises in computing transition probabilities when the…

Multiresolution Matrix Factorization (MMF) was recently introduced as a method for finding multiscale structure and defining wavelets on graphs/matrices. In this paper we derive pMMF, a parallel algorithm for computing the MMF…

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is…

Consider the continuous-time Markov Branching Process. In critical case we consider a situation when the generating function of intensity of transformation of particles has the infinite second moment, but its tail regularly varies in sense…

In this note, a general approach to the study of non-stationary Markov chains with catastrophes and the corresponding queuing models is considered, as well as to obtain estimates of the limiting regime itself. As an illustration, an example…

Deep architecture such as hierarchical semi-Markov models is an important class of models for nested sequential data. Current exact inference schemes either cost cubic time in sequence length, or exponential time in model depth. These costs…

We consider a finite state discrete time process X. Without loss of generality the finite state space can be identified with the set of unit vectors {e1, e2, . . . , eN} with ei = (0, . . . , 0, 1, 0, . . . , 0)0 2 RN. For a Markov chain…

In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition…

In multi-state life insurance, an adequate balance between analytic tractability, computational efficiency, and statistical flexibility is of great importance. This might explain the popularity of Markov chain modelling, where matrix…

This paper develops a framework for quantile regression in binary longitudinal data settings. A novel Markov chain Monte Carlo (MCMC) method is designed to fit the model and its computational efficiency is demonstrated in a simulation…

The bike sharing systems are fast increasing as a public transport mode in urban short trips, and have been developed in many major cities around the world. A major challenge in the study of bike sharing systems is that large-scale and…