Related papers: Markov chains under nonlinear expectation
This paper is motivated by examples from stochastic reaction network theory. The $Q$-matrix of a stochastic reaction network can be derived from the reaction graph, an edge-labelled directed graph encoding the jump vectors of an associated…
The main target of this paper is to present the Markov chain C that, not giving explicitly the queue lengths stationary probabilities, has the necessary information to its determination for open networks of queues with several classes of…
Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…
A method of constructing Markov chains on finite state spaces is provided. The chain is specified by three constraints: stationarity, dependence and marginal distributions. The generalized Pythagorean theorem in information geometry plays a…
The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…
In multi-state life insurance, an adequate balance between analytic tractability, computational efficiency, and statistical flexibility is of great importance. This might explain the popularity of Markov chain modelling, where matrix…
The class of nonlinear Markov processes is characterized by the dependence of the current state of the process on its current distribution in addition to the dependence on the previous state. Due to this feature, these processes are…
A noncommutative Fornasini-Marchesini system (a multi-variable version of a linear system) can be realized within a weak Markov process (a model for quantum evolution). For a discrete time parameter the resulting structure is worked out…
Any maximal monotone operator can be characterized by a convex function. The family of such convex functions is invariant under a transformation connected with the Fenchel-Legendre conjugation. We prove that there exist a convex…
We consider Markov decision processes where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies involve the optimisation of observation times as well as the subsequent action values. We…
We consider a regulated multi-class instantaneous matching system with reneging, in which each event requires $K \geq 2$ distinct impatient agents who wait in their respective queues. Each agent class is subject to a buffer capacity,…
We study the large deviations of Markov chains under the sole assumption that the state space is discrete. In particular, we do not require any of the usual irreducibility and exponential tightness assumptions. Using subadditive arguments,…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
This paper studies the long-term behaviour of a continuous time Markov chain formed by two non-negative integer valued components that evolve subject to a competitive interaction. In the absence of interaction the Markov chain is just a…
We study a class of Markov chains that model the evolution of a quantum system subject to repeated measurements. Each Markov chain in this class is defined by a measure on the space of matrices. It is then given by a random product of…
We look at a stochastic time-varying optimization problem and we formulate online algorithms to find and track its optimizers in expectation. The algorithms are derived from the intuition that standard prediction and correction steps can be…
$M_n(\mathbb{C})$ denotes the set of $n$ by $n$ complex matrices. Consider continuous time quantum semigroups $\mathcal{P}_t= e^{t\, \mathcal{L}}$, $t \geq 0$, where $\mathcal{L}:M_n(\mathbb{C}) \to M_n(\mathbb{C})$ is the infinitesimal…
Performing numerical integration when the integrand itself cannot be evaluated point-wise is a challenging task that arises in statistical analysis, notably in Bayesian inference for models with intractable likelihood functions. Markov…
Certain linear matrix operators arise naturally in systems analysis and design problems involving cascade interconnections of linear time-invariant systems, including problems of stabilization, estimation, and model order reduction. We…
We consider a general $k$ dimensional discounted infinite server queues process (alternatively, an Incurred But Not Reported (IBNR) claim process) where the multivariate inputs (claims) are given by a $k$ dimensional finite state Markov…