Related papers: Some characterizations for Markov processes as mix…
Many random combinatorial objects have a component structure whose joint distribution is equal to that of a process of mutually independent random variables, conditioned on the value of a weighted sum of the variables. It is interesting to…
We study very simple sorting algorithms based on a probabilistic comparator model. In our model, errors in comparing two elements are due to (1) the energy or effort put in the comparison and (2) the difference between the compared…
Some $M|G|\infty$ queue systems parameters values approximations, obtained through the consideration of an adequate Markov renewal process, are presented, and studied.
Relational Markov Decision Processes are a useful abstraction for complex reinforcement learning problems and stochastic planning problems. Recent work developed representation schemes and algorithms for planning in such problems using the…
Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…
The paper considers very general multivariate modifications of Cramer-Lundberg risk model. The claims can be of different types and can arrive in groups. The groups arrival processes within a type have constant intensities. The counting…
We describe a new construction of a family of measures on a group with the same Poisson boundary. Our approach is based on applying Markov stopping times to an extension of the original random walk.
We introduce and study a simple Markovian model of random separable permutations. Our first main result is the almost sure convergence of these permutations towards a random limiting object in the sense of permutons, which we call the…
Markov jump processes (MJPs) are continuous-time stochastic processes widely used in a variety of applied disciplines. Inference for MJPs typically proceeds via Markov chain Monte Carlo, the state-of-the-art being a uniformization-based…
Monoid actions of trace monoids over finite sets are powerful models of concurrent systems---for instance they encompass the class of 1-safe Petri nets. We characterise Markov measures attached to concurrent systems by finitely many…
We construct a family of self-similar Markov martingales with given marginal distributions. This construction uses the self-similarity and Markov property of a reference process to produce a family of Markov processes that possess the same…
A multivariate fractional Poisson process was recently defined in Beghin and Macci (2016) by considering a common independent random time change for a finite dimensional vector of independent (non-fractional) Poisson processes; moreover it…
When two Markov operators commute, it suggests that we can couple two copies of one of the corresponding processes. We explicitly construct a number of couplings of this type for a commuting family of Markov processes on the set of…
In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
We define a Markov process on the partitions of $[n]=\{1,\ldots,n\}$ by drawing a sample in $[n]$ at each time of a Poisson process, by merging blocks that contain one of these points and by leaving all other blocks unchanged. This…
The main purpose of this paper is to consider the multiple birth properties for multi-type Markov branching processes. We first construct a new multi-dimensional Markov process based on the multi-type Markov branching process, which can…
We consider Markov chains on general state spaces in stationary random environment which are defined by a random mapping that is contractive up to a bounded perturbation. We prove their convergence to a limiting law, providing convergence…
We introduce the BMRMM package implementing Bayesian inference for a class of Markov renewal mixed models which can characterize the stochastic dynamics of a collection of sequences, each comprising alternative instances of categorical…