Related papers: The Markov chain embedding problem in a low jump f…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…
A continuous-time random walk in the quarter plane with homogeneous transition rates is considered. Given a non-negative reward function on the state space, we are interested in the expected stationary performance. Since a direct derivation…
The problem of structured matrix estimation has been studied mostly under strong noise dependence assumptions. This paper considers a general framework of noisy low-rank-plus-sparse matrix recovery, where the noise matrix may come from any…
We address the problem of estimating the mixing time $t_{\mathsf{mix}}$ of an arbitrary ergodic finite-state Markov chain from a single trajectory of length $m$. The reversible case was addressed by Hsu et al. [2019], who left the general…
Many biological and medical questions can be modeled using time-to-event data in finite-state Markov chains, with the phase-type distribution describing intervals between events. We solve the inverse problem: given a phase-type…
We consider the problem of bounding mean first passage times for a class of continuous-time Markov chains that captures stochastic interactions between groups of identical agents. The quantitative analysis of such probabilistic population…
Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…
Spectral methods have proven to be a highly effective tool in understanding the intrinsic geometry of a high-dimensional data set $\left\{x_i \right\}_{i=1}^{n} \subset \mathbb{R}^d$. The key ingredient is the construction of a Markov chain…
Stochastic transitions between discrete microscopic states play an important role in many physical and biological systems. Often, these transitions lead to fluctuations on a macroscopic scale. A classic example from neuroscience is the…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…
A statistical test is presented to decide whether data are adequately described by probabilistic functions of finite state Markov chains (''hidden Markov models'') as applied in the analysis of ion channel data. Particularly, the test can…
Modeling the dynamics of non-stationary stochastic systems requires balancing the representational power of deep learning with the mathematical transparency of classical models. While classical Markov transition operators provide explicit,…
The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…
Motivated by a model presented by S. Gudder, we study a quantum generalization of Markov chains and discuss the relation between these maps and open quantum random walks, a class of quantum channels described by S. Attal et al. We consider…
For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…
In order to approximate a continuous time stochastic process by discrete time Markov chains one has several options to embed the Markov chains into continuous time processes. On the one hand there is the Markov embedding, which uses…
We investigate multivariate regular variation in the context of time-homogeneous Markov chains on general vector spaces and in random coefficient linear models. In the first part, we show that the regular variation of the stationary…
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…
The study of time-inhomogeneous Markov jump processes is a traditional topic within probability theory that has recently attracted substantial attention in various applications. However, their flexibility also incurs a substantial…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…