Related papers: Random-step Markov processes
Complex systems may often be characterized by their hierarchical dynamics. In this paper do we present a method and an operational algorithm that automatically infer this property in a broad range of systems; discrete stochastic processes.…
The investigation of random walks is central to a variety of stochastic processes in physics, chemistry, and biology. To describe a transport phenomenon, we study a variant of the one-dimensional persistent random walk, which we call a…
We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit…
In many applications, the common assumption that a driving noise process affecting a system is independent or Markovian may not be realistic, but the noise process may be assumed to be stationary. To study such problems, this paper…
We consider two classes of irreducible Markovian arrival processes specified by the matrices $C$ and $D$. The Markov Modulated Poison Process (MMPP) and the Markovian Switched Poison Process (MSPP). The former exhibits a diagonal $D$ while…
We present a new statistical test to detect that a presented state of a reversible Markov chain was not chosen from a stationary distribution. In particular, given a value function for the states of the Markov chain, we would like to…
Multistate Markov models are a canonical parametric approach for data modeling of observed or latent stochastic processes supported on a finite state space. Continuous-time Markov processes describe data that are observed irregularly over…
A state-dependent 1-limited polling model with N queues is analyzed. The routing strategy generalizes the classical Markovian polling model, in the sense that two routing matrices are involved, the choice being made according to the state…
Here, a new two-dimensional process, discrete in time and space, that yields the results of both a random walk and a quantum random walk, is introduced. This model describes the population distribution of four coin states |1>,-|1>, |0> -|0>…
In this paper we address the statistical problem of testing if a stationary process is Gaussian. The observation consists in a finite sample path of the process. Using a random projection technique introduced and studied in Cuesta-Albertos…
Numerical Generalized Randomized Hamiltonian Monte Carlo is introduced, as a robust, easy to use and computationally fast alternative to conventional Markov chain Monte Carlo methods for continuous target distributions. A wide class of…
It is shown that large deviation statistical quantities of the discrete time, finite state Markov process $P_{n+1}^{(j)}=\sum_{k=1}^NH_{jk}P_n^{(k)}$, where P_n^{(j)} is the probability for the j-state at the time step n and H_{jk} is the…
We are concerned with the absolute continuity of stationary distributions corresponding to some piecewise deterministic Markov process, being typically encountered in biological models. The process under investigation involves a…
The act of a person juggling can be viewed as a Markov process if we assume that the juggler throws to random heights. I make this association for the simplest reasonable model of random juggling and compute the steady state probabilities…
Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…
The extremes of a univariate Markov chain with regulary varying stationary marginal distribution and asymptotically linear behavior are known to exhibit a multiplicative random walk structure called the tail chain. In this paper, we extend…
Dealing with unichain MDPs, we consider stationary distributions of policies that coincide in all but $n$ states. In these states each policy chooses one of two possible actions. We show that the stationary distributions of n+1 such…
In multi-period stochastic optimization problems, the future optimal decision is a random variable whose distribution depends on the parameters of the optimization problem. We analyze how the expected value of this random variable changes…
We propose Markov two-components processes (M2CP) as a probabilistic model of asynchronous systems based on the trace semantics for concurrency. Considering an asynchronous system distributed over two sites, we introduce concepts and tools…
Brownian motion whose infinitesimal variance changes according to a three-state continuous time Markov Chain is studied. This Markov Chain can be viewed as a telegraph process with one on state and two off states. We first derive the…