Related papers: Nonlinear Markov Chains with Finite State Space: I…
Many applications in networked control require intermittent access of a controller to a system, as in event-triggered systems or information constrained control applications. Motivated by such applications and extending previous work on…
Switching ARMA models greatly enhance the standard linear models to the extent that different ARMA model is allowed in a different regime, and the regime switching is typically assumed a Markov chain on the finite states of potential…
We establish convergence to an invariant measure as time tends to infinity, for a large class of (possibly non-Markovian) stochastic volatility models. Our arguments are based on a novel coupling idea for Markov chains which also extends to…
We prove that for a random walk on the real line whose increments have zero mean and are either integer-valued or spread out (i.e. the distributions of the steps of the walk are eventually non-singular), the Markov chain of overshoots above…
We provide a nonasymptotic analysis of convergence to stationarity for a collection of Markov chains on multivariate state spaces, from arbitrary starting points, thereby generalizing results in [Khare and Zhou Ann. Appl. Probab. 19 (2009)…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
We consider finite-state time-nonhomogeneous Markov chains where the probability of moving from state $i$ to state $j\neq i$ at time $n$ is $G(i,j)/n^\zeta$ for a ``generator'' matrix $G$ and strength parameter $\zeta>0$. In these chains,…
Mathematical models of motility are often based on random-walk descriptions of discrete individuals that can move according to certain rules. It is usually the case that large masses concentrated in small regions of space have a great…
The equilibrium distributions of a Markovian model describing the interaction of several classes of permanent connections in a network are analyzed. It has been introduced by Graham and Robert. For this model each of the connections has a…
Motivated by techniques developed in recent progress on lower bounds for sublinear time algorithms (Behnezhad, Roghani and Rubinstein, STOC 2023, FOCS 2023, and STOC 2024) we introduce and study a new class of randomized algorithmic…
We study branching Markov chains on a countable state space (space of types) $\mathscr{X}$, with the focus on the qualitative aspects of the limit behaviour of the evolving empirical population distributions. No conditions are imposed on…
The asymptotic behavior of the quantum walk on the line is investigated focusing on the probability distribution of chirality independently of position. The long-time limit of this distribution is shown to exist and to depend on the initial…
In this work, we characterise the statistics of Markov chains by constructing an associated sequence of periodic differential operators. Studying the density of states of these operators reveals the absolutely continuous invariant measure…
This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…
We investigate recurrence and transience of Branching Markov Chains (BMC) in discrete time. Branching Markov Chains are clouds of particles which move (according to an irreducible underlying Markov Chain) and produce offspring…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We propose an analytic approach for the steady-state dynamics of Markov processes on locally tree-like graphs. It is based on time-translation invariant probability distributions for edge trajectories, which we encode in terms of infinite…
We study stochastic pairwise interaction network systems whereby a finite population of agents, identified with the nodes of a graph, update their states in response to both individual mutations and pairwise interactions with their…
For Markov chains with a partially ordered finite state space we show strong stationary duality under the condition of M\"obius monotonicity of the chain. We show relations of M\"obius monotonicity to other definitions of monotone chains.…
Using Foster-Lyapunov techniques we establish new conditions on non-extinction, non-explosion, coming down from infinity and staying infinite, respectively, for the general continuous-state nonlinear branching processes introduced in Li et…