Related papers: Qualitative Reachability for Open Interval Markov …
If the conditional information of a classical probability distribution of three random variables is zero, then it obeys a Markov chain condition. If the conditional information is close to zero, then it is known that the distance (minimum…
This article provides the first procedure for computing a fully data-dependent interval that traps the mixing time $t_{\text{mix}}$ of a finite reversible ergodic Markov chain at a prescribed confidence level. The interval is computed from…
Despite its prevalence, probabilistic bisimilarity suffers from a lack of robustness under minuscule perturbations of the transition probabilities. This can lead to discontinuities in the probabilistic bisimilarity distance function,…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We introduce the avoidance Markov metrics and theories which provide more flexibility in the design of random walk and impose new conditions on the walk to avoid (or transit) a specific node (or a set of nodes) before the stopping criteria.…
This work introduces a notion of approximate probabilistic trace equivalence for labelled Markov chains, and relates this new concept to the known notion of approximate probabilistic bisimulation. In particular this work shows that the…
Computing reachability probabilities is a fundamental problem in the analysis of probabilistic programs. This paper aims at a comprehensive and comparative account on various martingale-based methods for over- and under-approximating…
We consider discrete-time Markov decision processes in which the decision maker is interested in long but finite horizons. First we consider reachability objective: the decision maker's goal is to reach a specific target state with the…
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…
In this paper we consider the field of local times of a discrete-time Markov chain on a general state space, and obtain uniform (in time) upper bounds on the total variation distance between this field and the one of a sequence of $n$…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
Empirical likelihood is a powerful semi-parametric method increasingly investigated in the literature. However, most authors essentially focus on an i.i.d. setting. In the case of dependent data, the classical empirical likelihood method…
A new approach for optimal estimation of Markov chains with sparse transition matrices is presented.
We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other…
We show how to construct a topological Markov map of the interval whose invariant probability measure is the stationary law of a given stochastic chain of infinite order. In particular we caracterize the maps corresponding to stochastic…
We develop an algorithm for computing bounded reachability probability for hybrid systems, i.e., the probability that the system reaches an unsafe region within a finite number of discrete transitions. In particular, we focus on hybrid…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available.…
The approximate uniform sampling of graph realizations with a given degree sequence is an everyday task in several social science, computer science, engineering etc. projects. One approach is using Markov chains. The best available current…
This paper presents optimizations to improve the scalability of reachability analysis on a subclass of hybrid automata extended with stochasticity. The optimizations target different components of the analysis, such as quantifier…