Related papers: Exit Times for a Discrete Markov Additive Process

This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…

Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…

We consider a Markov process $X$, which is the solution of a stochastic differential equation driven by a L\'{e}vy process $Z$ and an independent Wiener process $W$. Under some regularity conditions, including non-degeneracy of the…

The discrete class algorithm presented in this paper is an efficient simulation tool for stochastic processes governed by a reasonably small set of transition rates. The algorithm is presented, its performance compared to prevailing methods…

Modern methods of simulating molecular systems are based on the mathematical theory of Markov operators with a focus on autonomous equilibrated systems. However, non-autonomous physical systems or non-autonomous simulation processes are…

We consider a class of piecewise-deterministic Markov processes where the state evolves according to a linear dynamical system. This continuous time evolution is interspersed by discrete events that occur at random times and change (reset)…

Markov chains have long been used for generating random variates from spatial point processes. Broadly speaking, these chains fall into two categories: Metropolis-Hastings type chains running in discrete time and spatial birth-death chains…

Deriving the time-dependent expected reward function associated with a continuous-time Markov chain involves the computation of its transient deviation matrix. In this paper we focus on the special case of a finite quasi-birth-and-death…

In this paper we introduce a bivariate distribution on $\mathbb{R}_{+} \times \mathbb{N}$ arising from a single underlying Markov jump process. The marginal distributions are phase-type and discrete phase-type distributed, respectively,…

Motivated by discrete kinetic models for non-cooperative molecular motors on periodic tracks, we consider random walks (also not Markov) on quasi one dimensional (1d) lattices, obtained by gluing several copies of a fundamental graph in a…

We derive the discrete linear systems associated to multi--matrix models, the corresponding discrete hierarchies and the appropriate coupling conditions. We also obtain the $W_{1+\infty}$ constraints on the partition function. We then apply…

In the present Short Note an idea is proposed to explain the emergence and the observation of processes in complex media that are driven by fractional non-Markovian master equations. Particle trajectories are assumed to be solely Markovian…

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…

In this paper, we propose a novel stochastic process that serves as a natural discrete-time counterpart to the continuous-time model known as the ``Poisson hyperbolic staircase'' proposed by Levikson et al. (1999), and clarify its…

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…

We present preliminary results of the investigation of the properties of the Markov random walk in the energy space generated by the Wang-Landau probability. We build transition matrix in the energy space (TMES) using the exact density of…

A two-parameter family of discrete-time exactly-solvable exclusion processes on a one-dimensional lattice is introduced, which contains the asymmetric simple exclusion process and the drop-push model as particular cases. The process is…

We study theoretical and empirical aspects of the mean exit time of financial time series. The theoretical modeling is done within the framework of continuous time random walk. We empirically verify that the mean exit time follows a…

This paper is concerned with a compositional approach for constructing finite Markov decision processes of interconnected discrete-time stochastic control systems. The proposed approach leverages the interconnection topology and a notion of…

We consider the problem of `discrete-time persistence', which deals with the zero-crossings of a continuous stochastic process, X(T), measured at discrete times, T = n(\Delta T). For a Gaussian Stationary Process the persistence (no…