Related papers: Random-step Markov processes
The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…
We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…
Consider a first-order autoregressive process $X_i=\beta X_{i-1}+\varepsilon_i,$ where $\varepsilon_i=G(\eta_i,\eta_{i-1},\ldots)$ and $\eta_i,i\in\mathbb{Z}$ are i.i.d. random variables. Motivated by two important issues for the inference…
Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be…
Given a sequence of i.i.d. random functions $\Psi_{n}:\mathbb{R}\to\mathbb{R}$, $n\in\mathbb{N}$, we consider the iterated function system and Markov chain which is recursively defined by $X_{0}^{x}:=x$ and…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…
We consider a class of stochastic dynamical systems, called piecewise deterministic Markov processes, with states $(x, \s)\in \O\times \G$, $\O$ being a region in $\bbR^d$ or the $d$--dimensional torus, $\G$ being a finite set. The…
We compute the stationary distribution of a continuous-time Markov chain which is constructed by gluing together two finite, irreducible Markov chains by identifying a pair of states of one chain with a pair of states of the other and…
A nonlinear Markov chain is a discrete time stochastic process whose transitions depend on both the current state and the current distribution of the process. The nonlinear Markov chain over a infinite state space can be identified by a…
A theory of systems with long-range correlations based on the consideration of binary N-step Markov chains is developed. In our model, the conditional probability that the i-th symbol in the chain equals zero (or unity) is a linear function…
Let $X_1, X_2,\ldots$ be random elements of the Skorokhod space $D(\mathbb{R})$ and $\xi_1, \xi_2, \ldots$ positive random variables such that the pairs $(X_1,\xi_1), (X_2,\xi_2),\ldots$ are independent and identically distributed. We call…
Suppose that $(X_t)_{t \ge 0}$ is a one-dimensional Brownian motion with negative drift $-\mu$. It is possible to make sense of conditioning this process to be in the state $0$ at an independent exponential random time and if we kill the…
We construct a stochastic fluid process with an underlying piecewise deterministic Markov process (PDMP) akin to the one used in the construction of the rational arrival process (RAP), which we call the RAP-modulated fluid process. As…
We define the concept of an `open' Markov process, a continuous-time Markov chain equipped with specified boundary states through which probability can flow in and out of the system. External couplings which fix the probabilities of…
An infinite system of point particles placed in $\mathds{R}^d$ is studied. Its constituents perform random jumps with mutual repulsion described by a translation-invariant jump kernel and interaction potential, respectively. The pure states…
Markov jump processes are continuous-time stochastic processes with a wide range of applications in both natural and social sciences. Despite their widespread use, inference in these models is highly non-trivial and typically proceeds via…
We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…
Random processes with stationary increments and intrinsic random processes are two concepts commonly used to deal with non-stationary random processes. They are broader classes than stationary random processes and conceptually closely…
The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of…