Related papers: Interruptible exact sampling in the passive case

Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require $\tilde{O}(\tau/\pi(v))$ operations to approximate the…

We consider Markov chains on general state spaces in stationary random environment which are defined by a random mapping that is contractive up to a bounded perturbation. We prove their convergence to a limiting law, providing convergence…

Drawing on some recent results that provide the formalism necessary to definite stationarity for infinite random graphs, this paper initiates the study of statistical and learning questions pertaining to these objects. Specifically, a…

For a class of irreducible Markov chains with an infinitely countable set of states, we establish a new verifiable necessary and sufficient condition for recurrence and transience. We show that if one of the basic assumptions is not…

We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing…

Markov chains are a convenient means of generating realizations of networks, since they require little more than a procedure for rewiring edges. If a rewiring procedure exists for generating new graphs with specified statistical properties,…

For discrete-time linear systems subject to parametric uncertainty described by random variables, we develop a sampling-based Stochastic Model Predictive Control algorithm. Unlike earlier results employing a scenario approximation, we…

Markov chain Monte Carlo is a widely-used technique for generating a dependent sequence of samples from complex distributions. Conventionally, these methods require a source of independent random variates. Most implementations use…

In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is…

In this paper, we develop methods of nonlinear filtering and prediction of an unobservable Markov chain with a finite set of states. This Markov chain controls coefficients of AR(p) model. Using observations generated by AR(p) model we have…

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…

Consider a Markov chain with finite state $\{0, 1, ..., d\}$. We give the generation functions (or Laplace transforms) of absorbing (passage) time in the following two situations : (1) the absorbing time of state $d$ when the chain starts…

A fixed-confidence, finite-sample problem of active hypothesis testing arises in many safety-critical applications. Situated in the context of sequential hypothesis testing, this paper studies the effect of hypothesis elimination on the…

We derive a sufficient condition for a $k$-th order homogeneous Markov chain $\mathbf{Z}$ with finite alphabet $\mathcal{Z}$ to have a unique invariant distribution on $\mathcal{Z}^k$. Specifically, let $\mathbf{X}$ be a first-order,…

This note presents a simple proof of the monotonicity of the invariant distribution of a discrete Markov chain with a finite state space. This answers a question recently raised by David Siegmund.

It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…

General Markov chains in an arbitrary phase space are considered in the framework of the operator treatment. Markov operators continue from the space of countably additive measures to the space of finitely additive measures. Cycles of…

We describe an exact test of the null hypothesis that a Markov chain is nth order versus the alternate hypothesis that it is $(n+1)$-th order. The procedure does not rely on asymptotic properties, but instead builds up the test statistic…

For many probability distributions of interest, it is quite difficult to obtain samples efficiently. Often, Markov chains are employed to obtain approximately random samples from these distributions. The primary drawback to traditional…

Markov Chain Monte Carlo (MCMC) algorithms are routinely used to draw samples from distributions with intractable normalization constants. However, standard MCMC algorithms do not apply to doubly-intractable distributions in which there are…