Related papers: Some characterizations for Markov processes as mix…
We consider random processes that are history-dependent, in the sense that the distribution of the next step of the process at any time depends upon the entire past history of the process. In general, therefore, the Markov property cannot…
Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous state space explosion problem. We alleviate this problem by…
This chapter is an attempt to present a mathematical theory of compound fractional Poisson processes. The chapter begins with the characterization of a well-known L\'evy process: The compound Poisson process. The semi-Markov extension of…
Comparison results for Markov processes w.r.t. function class induced (integral) stochastic orders have a long history. The most general results so far for this problem have been obtained based on the theory of evolution systems on Banach…
If a given aggregate process $S$ is a compound mixed Poisson process under a probability measure $P$, a characterization of all probability measures $Q$ on the domain of $P$, such that $P$ and $Q$ are progressively equivalent and $S$…
Representations of branching Markov processes and their measure-valued limits in terms of countable systems of particles are constructed for models with spatially varying birth and death rates. Each particle has a location and a "level,"…
Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…
Stacy distribution defined for the first time in 1961 provides a flexible framework for modelling of a wide range of real-life behaviours. It appears under different names in the scientific literature and contains many useful particular…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
This note make the observation that property A for a space is equivalent to the existence of a Markov process on the space which has a (uniformly) trivial Poisson boundary.
Semi-Markov processes are Markovian processes in which the firing time of the transitions is modelled by probabilistic distributions over positive reals interpreted as the probability of firing a transition at a certain moment in time. In…
Changing time of simple continuous-time Markov counting processes by independent unit-rate Poisson processes results in Markov counting processes for which we provide closed-form transition rates via composition of trajectories and with…
Generalization of the Lorden's inequality is an excellent tool for obtaining strong upper bounds for the convergence rate for various complicated stochastic models. This paper demonstrates a method for obtaining such bounds for some…
We study a two parameter family of processes with linear regressions and linear conditional variances. We give conditions for the unique solution of this problem, and point out the connection between the resulting Markov processes and the…
The embedding problem of Markov transition matrices into continuous-time Markov semigroups is a classic problem that regained a lot of impetus and activities in recent years. We consider it here for the following generalisation of the…
In this paper, we develop some matrix Poisson's equations satisfied by the mean and variance of the mixing time in an irreducible positive-recurrent discrete-time Markov chain with infinitely-many levels, and provide a computational…
Inference for partially observed Markov process models has been a longstanding methodological challenge with many scientific and engineering applications. Iterated filtering algorithms maximize the likelihood function for partially observed…
We consider a type of Markov property for set-indexed processes which is satisfied by all processes with independent increments and which allows us to introduce a transition system theory leading to the construction of the process. A…
This paper studies parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a finite set of parameters. Fixing values for all parameters…
We study Markov-modulated affine processes (abbreviated MMAPs), a class of Markov processes that are created from affine processes by allowing some of their coefficients to be a function of an exogenous Markov process. MMAPs allow for…