Related papers: Some notes on a semi-Markov matrix occurring in th…
We study Markov-modulated affine processes (abbreviated MMAPs), a class of Markov processes that are created from affine processes by allowing some of their coefficients to be a function of an exogenous Markov process. MMAPs allow for…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…
We introduce and study some queueing models with random resetting, including Markovian and non--Markovian models. The Markovian models include M/M/$\infty$, M/M/r and M/M/1+M queues with random resetting, in which a continuous-time Markov…
Markov-modulated fluid queues are popular stochastic processes frequently used for modelling real-life applications. An important performance measure to evaluate in these applications is their steady-state behaviour, which is determined by…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
The embedding problem of Markov transition matrices into continuous-time Markov semigroups is a classic problem that regained a lot of impetus and activities in recent years. We consider it here for the following generalisation of the…
For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the…
We consider the Markov renewal equation $F(t) = f(t) + \boldsymbol{\mu}*F(t)$ for vector-valued functions $f,F: \mathbb{R} \to \mathbb{R}^{p}$ and a $p \times p$ matrix $\boldsymbol{\mu}$ of locally finite measures $\mu^{i,j}$ on…
Markov decision processes (MDPs) in queues and networks have been an interesting topic in many practical areas since the 1960s. This paper provides a detailed overview on this topic and tracks the evolution of many basic results. Also, this…
We consider the problem of constructing an appropriate multivariate model for the study of the counterparty credit risk in credit rating migration problem. For this financial problem different multivariate Markov chain models were proposed.…
Quadratic matrix equations of the kind $A_1X^2+A_0X+A_{-1}=X$ are encountered in the analysis of Quasi--Birth-Death stochastic processes where the solution of interest is the minimal nonnegative solution $G$. In many queueing models,…
In this paper we propose a semi-Markov modulated model of interest rates. We assume that the switching process is a semi-Markov process with finite state space E and the modulated process is a diffusive process. We derive recursive…
In the Markov decision process model, policies are usually evaluated by expected cumulative rewards. As this decision criterion is not always suitable, we propose in this paper an algorithm for computing a policy optimal for the quantile…
The aim of this paper is to study some continuous-time bivariate Markov processes arising from group representation theory. The first component (level) can be either discrete (quasi-birth-and-death processes) or continuous (switching…
A semi-Markov process is one that changes states in accordance with a Markov chain but takes a random amount of time between changes. We consider the generalisation to semi-Markov processes of the classical Lamperti law for the occupation…
Markov jump process models have many applications across science. Often, these models are defined on a state-space of product form and only one of the components of the process is of direct interest. In this paper, we extend the marginal…
We propose a matrix model which embodies the semiclassical approach to the problem of quantum transport in chaotic systems. Specifically, a matrix integral is presented whose perturbative expansion satisfies precisely the semiclassical…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured…
This paper shows a novel way of simulating a Markov process by a quantum computer. The main purpose of the paper is to show a particular application of quantum computing in the field of stochastic processes analysis. Using a Quantum…
The computation of matrix functions is a well-studied problem. Of special importance are the exponential and the logarithm of a matrix, where the latter also raises existence and uniqueness questions. This is particularly relevant in the…