Related papers: Sequential One-Sided Hypothesis Testing of Markov …

Hybrid systems whose mode dynamics are governed by non-linear ordinary differential equations (ODEs) are often a natural model for biological processes. However such models are difficult to analyze. To address this, we develop a…

Markov chain Monte Carlo is a widely-used technique for generating a dependent sequence of samples from complex distributions. Conventionally, these methods require a source of independent random variates. Most implementations use…

A classical problem for Markov chains is determining their stationary (or steady-state) distribution. This problem has an equally classical solution based on eigenvectors and linear equation systems. However, this approach does not scale to…

In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…

We propose to study unitary matrix ensembles defined in terms of unitary stochastic transition matrices associated with Markov processes on graphs. We argue that the spectral statistics of such an ensemble (after ensemble averaging) depends…

We propose approaches for testing implementations of Markov Chain Monte Carlo methods as well as of general Monte Carlo methods. Based on statistical hypothesis tests, these approaches can be used in a unit testing framework to, for…

A cornerstone of human statistical learning is the ability to extract temporal regularities / patterns from random sequences. Here we present a method of computing pattern time statistics with generating functions for first-order Markov…

We consider the parameter estimation of Markov chain when the unknown transition matrix belongs to an exponential family of transition matrices. Then, we show that the sample mean of the generator of the exponential family is an…

Researchers faced with a sequence of candidate model specifications must often choose the best specification that does not violate a testable identification assumption. One option in this scenario is sequential specification tests:…

In this study, we address the central issue of statistical inference for Markov jump processes using discrete time observations. The primary problem at hand is to accurately estimate the infinitesimal generator of a Markov jump process, a…

A nonlinear Markov chain is a discrete time stochastic process whose transitions depend on both the current state and the current distribution of the process. The nonlinear Markov chain over a infinite state space can be identified by a…

Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result,…

The partially asymmetric exclusion process (PASEP) is an important model from statistical mechanics which describes a system of interacting particles hopping left and right on a one-dimensional lattice of N sites. It is partially asymmetric…

Rate processes are simple and analytically tractable models for many dynamical systems which switch stochastically between a discrete set of quasi stationary states but they may also approximate continuous processes by coarse grained,…

Probabilistic model checking for systems with large or unbounded state space is a challenging computational problem in formal modelling and its applications. Numerical algorithms require an explicit representation of the state space, while…

We consider a Markov process in continuous time with a finite number of discrete states. The time-dependent probabilities of being in any state of the Markov chain are governed by a set of ordinary differential equations, whose dimension…

We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the often unrealistic assumption that precise probabilities are…

The standard approach to realize a quantum repeater relies upon probabilistic but heralded entangled state manipulations and the storage of quantum states while waiting for successful events. In the literature on this class of repeaters,…

Markov decision processes (MDP) are useful to model concurrent process optimisation problems, but verifying them with numerical methods is often intractable. Existing approximative approaches do not scale well and are limited to memoryless…

We develop a first-order (pseudo-)gradient approach for optimizing functions over the stationary distribution of discrete-time Markov chains (DTMC). We give insights into why solving this optimization problem is challenging and show how…