Related papers: Stability in Distribution of Randomly Perturbed Qu…

Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…

This paper is about the rate of convergence of the Markov chain $X_{n+1}=AX_{n}+B_{n}$ (mod $p$), where $A$ is an integer matrix with nonzero eigenvalues and ${B_{n}}_{n}$ is a sequence of independent and identically distributed integer…

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume…

We focus on the parametric estimation of the distribution of a Markov environment from the observation of a single trajectory of a one-dimensional nearest-neighbor path evolving in this random environment. In the ballistic case, as the…

Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…

This paper presents a model of asymmetric bifurcating autoregressive process with random coefficients. We couple this model with a Galton Watson tree to take into account possibly missing observations. We propose least-squares estimators…

This article contains new tools for studying the shape of the stationary distribution of sizes in a dynamic economic system in which units experience random multiplicative shocks and are occasionally reset. Each unit has a Markov-switching…

We consider the problem of defining and fitting models of autoregressive time series of probability distributions on a compact interval of $\mathbb{R}$. An order-$1$ autoregressive model in this context is to be understood as a Markov…

Let $\{\boldsymbol{X}_n\}$ be a discrete-time $d$-dimensional process on $\mathbb{Z}_+^d$ with a supplemental (background) process $\{J_n\}$ on a finite set and assume the joint process $\{\boldsymbol{Y}_n\}=\{(\boldsymbol{X}_n,J_n)\}$ to…

For a directed graph $G(V_n, E_n)$ on the vertices $V_n = \{1,2, \dots, n\}$, we study the distribution of a Markov chain $\{ {\bf R}^{(k)}: k \geq 0\}$ on $\mathbb{R}^n$ such that the $i$th component of ${\bf R}^{(k)}$, denoted…

This paper considers the problem of randomized influence maximization over a Markovian graph process: given a fixed set of nodes whose connectivity graph is evolving as a Markov chain, estimate the probability distribution (over this fixed…

This paper considers a class of non-Markovian discrete-time random processes on a finite state space {1,...,d}. The transition probabilities at each time are influenced by the number of times each state has been visited and by a fixed a…

We develop criteria for recurrence and transience of one-dimensional Markov processes which have jumps and oscillate between $+\infty$ and $-\infty$. The conditions are based on a Markov chain which only consists of jumps (overshoots) of…

This paper provides error analyses of the algorithms most commonly used for the evaluation of the Chebyshev polynomial of the first kind $T_N(x)$. Some of these algorithms are shown to be backward stable. This means that the computed value…

We study general properties for the family of stochastic processes with polynomial regression property, that is that every conditional moment of the process is a polynomial. It turns out that then there exists a family of polynomial…

Let $(X,\|.\|)$ be a Banach space. Let $C$ be a nonempty, bounded, closed, and convex subset of $X$ and $T: C \rightarrow C$ be a $G$-monotone nonexpansive mapping. In this work, it is shown that the Mann iteration sequence defined by…

We show that long-term memory effects, present in the chaotic dispersion process generated by a meandering jet model, can be nonetheless taken into account by a first order Markov process, provided that the states of the phase space…

We consider a stochastic process in which independent identically distributed random matrices are multiplied and where the Lyapunov exponent of the product is positive. We continue multiplying the random matrices as long as the norm,…

We consider the dynamical system described by the area--preserving standard mapping. It is known for this system that $P(t)$, the normalized number of recurrences staying in some given domain of the phase space at time $t$ (so-clled…

We revisit, in a self contained way, the Markov property on planar maps and decorated planar maps from three perspectives. First, we characterize the laws on these planar maps that satisfy both the Markov property and rerooting invariance,…