Related papers: Some notes on a semi-Markov matrix occurring in th…
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is…
The paper discusses the continuous-time Markov Branching Process allowing Immigration. We are considering a critical case for which the second moment of offspring law and the first moment of immigration law are possibly infinite. Assuming…
This work is a continuation of [Kalikaeva, MPRF, 23(2):225-240]. The object of study is ``Markov-up processes'' on $\mathbb Z_+$ and the moment of downcrossing a certain barrier. The processes considered in this paper differ from Markov…
We give a generalization of the ergodic theorem for semi-Markov linear-type processes. This generalization is proved for the case when a common support of distributions defining this process is not arithmetic. Also we give an uniform…
In the paper we consider a stochastic model which called Markov Q-processes that forms a continuous-time Markov population system. Markov Q-processes are defined as stochastic Markov branching processes with trajectories continuing in the…
Given noisy, partial observations of a time-homogeneous, finite-statespace Markov chain, conceptually simple, direct statistical inference is available, in theory, via its rate matrix, or infinitesimal generator, $\mathsf{Q}$, since $\exp…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
The main purpose of this work is to study self-similar branching Markov chains. First we will construct such a process. Then we will establish certain Limit Theorems using the theory of self-similar Markov processes.
We compare different analytical and numerical methods for studying the partitions of a finite system into fragments. We propose a new numerical method of exploring the partition space by generating the Markov chains of partitions based on…
A $M/M/1$ queue with catastrophes is a modified $M/M/1$ queue model for which, according to the times of a Poisson process, catastrophes occur leaving the system empty. In this work, we study a fractional $M/M/1$ queue with catastrophes,…
We present an approach for testing for the existence of continuous generators of discrete stochastic transition matrices. Typically, the known approaches to ascertain the existence of continuous Markov processes are based in the assumption…
Let $\mu$ = ($\mu$t)t$\in$R be any 1-parameter family of probability measures on R. Its quantile process (Gt)t$\in$R : ]0, 1[ $\rightarrow$ RR, given by Gt($\alpha$) = inf{x $\in$ R : $\mu$t(]--$\infty$, x]) > $\alpha$}, is not Markov in…
Explicit and exact results are obtained for the joint queue-length distribution for the two-level non-preemptive Markovian priority queue. Marginal distributions are derived for the general multi-level problem. The results are based on a…
We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means…
We study time-changed Markov processes to speed up the convergence of Markov chain Monte Carlo (MCMC) algorithms. The time-changed process is defined by adjusting the speed of time of a base process via a user-chosen, state-dependent…
The martingale comparison method is extended to derive comparison results for path-independent functions for general semimartingales. Our approach allows to dismiss with the Markovian assumption on one of the processes made in previous…
We consider multi-state capture-recapture-recovery data where observed individuals are recorded in a set of possible discrete states. Traditionally, the Arnason-Schwarz model has been fitted to such data where the state process is modeled…
We introduce Markov Decision Processing Networks (MDPNs) as a multiclass queueing network model where service is a controlled, finite-state Markov process. The model exhibits a decision-dependent service process where actions taken…
We introduce a framework for approximate analysis of Markov decision processes (MDP) with bounded-, unbounded-, and infinite-horizon properties. The main idea is to identify a "core" of an MDP, i.e., a subsystem where we provably remain…