Computing expected transition events in reducible Markov chains
Probability
2017-06-09 v2
Abstract
We present a closed-form, computable expression for the expected number of times any transition event occurs during the transient phase of a reducible Markov chain. Examples of events include time to absorption, number of visits to a state, traversals of a particular transition, loops from a state to itself, and arrivals to a state from a particular subset of states. We give an analogous expression for time-average events, which describe the steady-state behavior of reducible chains as well as the long-term behavior of irreducible chains.
Keywords
Cite
@article{arxiv.0806.1291,
title = {Computing expected transition events in reducible Markov chains},
author = {Brian D. Ewald and Jeffrey Humpherys and Jeremy West},
journal= {arXiv preprint arXiv:0806.1291},
year = {2017}
}
Comments
15 pages, 1 table. Added numerical analysis and changed formatting. Fixed several errors