Related papers: Asymptotically linear iterated function systems on…

In this article, we primarily propose a novel Bayesian characterization of stationary and nonstationary stochastic processes. In practice, this theory aims to distinguish between global stationarity and nonstationarity for both parametric…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…

In the paper, the law of the iterated logarithm for additive functionals of Markov chains is obtained under some weak conditions, which are weaker than the conditions of invariance principle of additive functionals of Markov chains in M.…

We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…

We study the asymptotic behaviour of Markov chains $(X_n,\eta_n)$ on $\mathbb{Z}_+ \times S$, where $\mathbb{Z}_+$ is the non-negative integers and $S$ is a finite set. Neither coordinate is assumed to be Markov. We assume a moments bound…

We introduce the concept of a Markov influence system (MIS) and analyze its dynamics. An MIS models a random walk in a graph whose edges and transition probabilities change endogenously as a function of the current distribution. This…

This paper generalizes the notion of stochastic order to a relation between probability measures over arbitrary measurable spaces. This generalization is motivated by the observation that for the stochastic ordering of two stationary Markov…

We study the connections existing between max-infinitely divisible distributions and Poisson processes from the point of view of functional analysis. More precisely, we derive functional identities for the former by using well-known results…

A theory of additive Markov chains with long-range memory, proposed earlier in Phys. Rev. E 68, 06117 (2003), is developed and used to describe statistical properties of long-range correlated systems. The convenient characteristics of such…

There is a well-established theory linking certain semi-Markov chains and continuous-time random walks to time-fractional equations and anomalous diffusion. In this work, we go beyond the semi-Markov framework by considering some…

A Markov Additive Process is a bi-variate Markov process $(\xi,J)=\big((\xi_t,J_t),t\geq0\big)$ which should be thought of as a multi-type L\'evy process: the second component $J$ is a Markov chain on a finite space $\{1,\ldots,K\}$, and…

We focus on a data sequence produced by repetitive quantum measurement on an internal hidden quantum system, and call it a hidden Markovian process. Using a quantum version of the Perron-Frobenius theorem, we derive novel upper and lower…

We study existence and uniqueness of the fixed points solutions of a large class of non-linear variable discounted transfer operators associated to a sequential decision-making process. We establish regularity properties of these solutions,…

In queuing theory and related problems, it is very important to know the numerical characteristics of an investigated system - both in stationary and non-stationary modes. In some cases, such characteristics can be calculated, but this is…

In this paper, we study Markov-dependent reflected autoregressive processes, and other related models the analysis of which results in a vector-valued fixed-point functional equation of a certain type. In queueing terms, such processes…

We extend Hoeffding's lemma to general-state-space and not necessarily reversible Markov chains. Let $\{X_i\}_{i \ge 1}$ be a stationary Markov chain with invariant measure $\pi$ and absolute spectral gap $1-\lambda$, where $\lambda$ is…

In the first part of this thesis, we study a Markov chain on $\mathbb{R}_+ \times S$, where $\mathbb{R}_+$ is the non-negative real numbers and $S$ is a finite set, in which when the $\mathbb{R}_+$-coordinate is large, the $S$-coordinate of…

This work considers the problem of learning the Markov parameters of a linear system from observed data. Recent non-asymptotic system identification results have characterized the sample complexity of this problem in the single and…

Population dynamics are often subject to random independent changes in the environment. For the two strategy stochastic replicator dynamic, we assume that stochastic changes in the environment replace the payoffs and variance. This is…