Related papers: Stability in Distribution of Randomly Perturbed Qu…
We obtain complementary recurrence and transience criteria for processes $X=(X_n)_{n \ge 0}$ with values in $\mathbb R^d_+$ fulfilling a non-linear equation $X_{n+1}=MX_n+g(X_n)+ \xi_{n+1}$. Here $M$ denotes a primitive matrix having…
We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such…
Given a sequence of i.i.d. random functions $\Psi_{n}:\mathbb{R}\to\mathbb{R}$, $n\in\mathbb{N}$, we consider the iterated function system and Markov chain which is recursively defined by $X_{0}^{x}:=x$ and…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
We explore two notions of stationary processes. The first is called a random-step Markov process in which the stationary process of states, $(X_i)_{i \in \mathbb{Z}}$ has a stationary coupling with an independent process on the positive…
We discuss complementary recurrence and transience criteria for stochastic processes $(X_n)_{n \ge 0}$ with values in the $d$-dimensional orthant $\mathbb R^d_+$ fulfilling a non-linear stochastic equation of the form $X_{n+1}=MX_n+g(X_n)+…
Let $(Y_n)$ be a sequence of i.i.d. real valued random variables. Reflected random walk $(X_n)$ is defined recursively by $X_0=x \ge 0$, $X_{n+1} = |X_n - Y_{n+1}|$. In this note, we study recurrence of this process, extending a previous…
We show that for a large class of marked point processes there exists a random measure m with the predictable representation property such that iterated integrals with respect to m span the space of square integrable random variables.
We consider the quadratic family of maps given by $f_{a}(x)=1-a x^2$ with $x\in [-1,1]$, where $a$ is a Benedicks-Carleson parameter. For each of these chaotic dynamical systems we study the extreme value distribution of the stationary…
Consider a first-order autoregressive process $X_i=\beta X_{i-1}+\varepsilon_i,$ where $\varepsilon_i=G(\eta_i,\eta_{i-1},\ldots)$ and $\eta_i,i\in\mathbb{Z}$ are i.i.d. random variables. Motivated by two important issues for the inference…
In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
A classical random walk $(S_t, t\in\mathbb{N})$ is defined by $S_t:=\displaystyle\sum_{n=0}^t X_n$, where $(X_n)$ are i.i.d. When the increments $(X_n)_{n\in\mathbb{N}}$ are a one-order Markov chain, a short memory is introduced in the…
It is shown that large deviation statistical quantities of the discrete time, finite state Markov process $P_{n+1}^{(j)}=\sum_{k=1}^NH_{jk}P_n^{(k)}$, where P_n^{(j)} is the probability for the j-state at the time step n and H_{jk} is the…
We introduce an harmonic analysis for iterated function systems (IFS) (X, mu) which is based on a Markov process on certain paths. The probabilities are determined by a weight function W on X. From W we define a transition operator R_W…
We characterize recurrence and transience of nonnegative multivariate autoregressive processes of order one with random contractive coefficient matrix, of subcritical multitype Galton-Watson branching processes in random environment with…
In parameter estimation problems one computes a posterior distribution over uncertain parameters defined jointly by a prior distribution, a model, and noisy data. Markov Chain Monte Carlo (MCMC) is often used for the numerical solution of…
Non-linear Hawkes processes with memory kernels given by the sum of Erlang kernels are considered. It is shown that their stability properties can be studied in terms of an associated class of piecewise deterministic Markov processes,…
Inference for partially observed Markov process models has been a longstanding methodological challenge with many scientific and engineering applications. Iterated filtering algorithms maximize the likelihood function for partially observed…
Let $\Phi_n$ be an i.i.d. sequence of Lipschitz mappings of $\R^d$. We study the Markov chain $\{X_n^x\}_{n=0}^\infty$ on $\R^d$ defined by the recursion $X_n^x = \Phi_n(X^x_{n-1})$, $n\in\N$, $X_0^x=x\in\R^d$. We assume that…