Related papers: Testing Markov Chains without Hitting
Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…
We study the problem of learning the Markov order in categorical sequences that represent paths in a network, i.e. sequences of variable lengths where transitions between states are constrained to a known graph. Such data pose challenges…
This paper studies three kinds of long-term behaviours, namely reachability, repeated reachability and persistence, of quantum Markov chains (qMCs). As a stepping-stone, we introduce the notion of bottom strongly connected component (BSCC)…
We describe an exact test of the null hypothesis that a Markov chain is nth order versus the alternate hypothesis that it is $(n+1)$-th order. The procedure does not rely on asymptotic properties, but instead builds up the test statistic…
This book is an introduction to quantum Markov chains and explains how this concept is connected to the question of how well a lost quantum mechanical system can be recovered from a correlated subsystem. To achieve this goal, we strengthen…
The cover time of a Markov chain on a finite state space is the expected time until all states are visited. We show that if the cover time of a discrete-time Markov chain with rational transitions probabilities is bounded, then it is a…
We study the question of identity testing for structured distributions. More precisely, given samples from a {\em structured} distribution $q$ over $[n]$ and an explicit distribution $p$ over $[n]$, we wish to distinguish whether $q=p$…
Clustering algorithms frequently require the number of clusters to be chosen in advance, but it is usually not clear how to do this. To tackle this challenge when clustering within sequential data, we present a method for estimating the…
Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…
We propose a simple criterion for non-Markovianity: a quantum master equation is non-Markovian if and only if its \textit{trajectory set} contains a \textit{self-intersecting trajectory} (defined in the main text). Since self-intersection…
We make use of matrix representations of completely positive maps in order to study open quantum dynamics on graphs, with emphasis on quantum walks and the associated trajectories obtained via a monitoring of the position. We discuss the…
An important task in machine learning and statistics is the approximation of a probability measure by an empirical measure supported on a discrete point set. Stein Points are a class of algorithms for this task, which proceed by…
Inference for continuous-time Markov chains (CTMCs) becomes challenging when the process is only observed at discrete time points. The exact likelihood is intractable, and existing methods often struggle even in medium-dimensional…
We study the verification of a finite continuous-time Markov chain (CTMC) C against a linear real-time specification given as a deterministic timed automaton (DTA) A with finite or Muller acceptance conditions. The central question that we…
We propose polynomial-time algorithms to minimise labelled Markov chains whose transition probabilities are not known exactly, have been perturbed, or can only be obtained by sampling. Our algorithms are based on a new notion of an…
We develop some sufficient conditions for the stochastic ordering between hitting times, in a fixed state, for two Markov chains. In particular, we focus attention on the so called \emph{skip-free} case. In the analysis of such a case, we…
In this paper, we consider the Markov-Chain Monte Carlo (MCMC) approach for random sampling of combinatorial objects. The running time of such an algorithm depends on the total mixing time of the underlying Markov chain and is unknown in…
In this paper, we present reversibility preserving operations on Markov chain transition matrices. Simple row and column operations allow us to create new reversible transition matrices and yield an easy method for checking a Markov chain…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
We present a new statistical test to detect that a presented state of a reversible Markov chain was not chosen from a stationary distribution. In particular, given a value function for the states of the Markov chain, we would like to…