Related papers: A Note on Exact State Visit Probabilities in Two-S…

We study a strongly Non-Markovian variant of random walk in which the probability of visiting a given site $i$ is a function $f$ of number of previous visits $v(i)$ to the site. If the probability is proportional to number of visits to the…

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

In the experimental determination of the population transfer efficiency between discrete states of a coherently driven quantum system it is often inconvenient to measure the population of the target state. Instead, after the interaction…

We study distributions of meeting times for finite symmetric Markov chains. For Markov kernels defined on large state spaces which satisfy certain weak inhomogeneity in return probabilities of points up to large numbers of steps, we obtain…

Markov cohort state-transition models have been the standard approach for simulating the prognosis of patients or, more generally, the life trajectories of individuals over a time period. Current approaches for estimating the variance of a…

Consider a discrete time, ergodic Markov chain with finite state space which is started from stationarity. Fill and Lyzinski (2014) showed that, in some cases, the hitting time for a given state may be represented as a sum of a geometric…

We study the problem of sequentially testing whether a given stochastic process is generated by a known Markov chain. Formally, given access to a stream of random variables, we want to quickly determine whether this sequence is a trajectory…

Let $(M,d,\mu)$ be a uniformly discrete metric measure space satisfying space homogeneous volume doubling condition. We consider discrete time Markov chains on $M$ symmetric with respect to $\mu$ and whose one-step transition density is…

For a Markov chain $\mathbf{X}=\{X_i,i=1,2,...,n\}$ with the state space $\{0,1\}$, the random variable $S:=\sum_{i=1}^nX_i$ is said to follow a Markov binomial distribution. The exact distribution of $S$, denoted $\mathcal{L}S$, is very…

Multi-state models are frequently applied for representing processes evolving through a discrete set of state. Important classes of multi-state models arise when transitions between states may depend on the time since entry into the current…

Many biological and medical questions can be modeled using time-to-event data in finite-state Markov chains, with the phase-type distribution describing intervals between events. We solve the inverse problem: given a phase-type…

Markov decision processes model systems subject to nondeterministic and probabilistic uncertainty. A plethora of verification techniques addresses variations of reachability properties, such as: Is there a scheduler resolving the…

A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…

This work investigates the almost sure stabilization of a class of regime-switching systems based on discrete-time observations of both continuous and discrete components. It develops Shao's work [SIAM J. Control Optim., 55(2017), pp.…

Parametric Markov chains (pMC) are used to model probabilistic systems with unknown or partially known probabilities. Although (universal) pMC verification for reachability properties is known to be coETR-complete, there have been efforts…

Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is…

We address the problem of approximating the posterior probability distribution of the fixed parameters of a state-space dynamical system using a sequential Monte Carlo method. The proposed approach relies on a nested structure that employs…

A state on a tripartite quantum system $A \otimes B \otimes C$ forms a Markov chain if it can be reconstructed from its marginal on $A \otimes B$ by a quantum operation from $B$ to $B \otimes C$. We show that the quantum conditional mutual…

We study quantum trajectories of collective atomic spin states of $N$ effective two-level atoms driven with laser and cavity fields. We show that interesting ``entangled-state cycles'' arise probabilistically when the (Raman) transition…

Nonlinear Markov chains with finite state space have been introduced in Kolokoltsov (2010). The characteristic property of these processes is that the transition probabilities do not only depend on the state, but also on the distribution of…