Related papers: Approximate Probabilistic Bisimulation for Continu…
Poisson's equation has a lot of applications in various areas. Usually it is hard to derive the explicit expression of the solution of Poisson's equation for a Markov chain on an infinitely many state space. We will present a computational…
We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting…
We consider a Markov chain approximation scheme for utility maximization problems in continuous time, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauss-Hermite approximation of the…
The utilization of model checking has been suggested as a formal verification technique for analyzing critical systems. However, the primary challenge in applying to complex systems is state space explosion problem. To address this issue,…
We have developed a notion of global bisimulation distance between processes which goes somehow beyond the notions of bisimulation distance already existing in the literature, mainly based on bisimulation games. Our proposal is based on the…
We propose a discrete time discrete space Markov chain approximation with a Brownian bridge correction for computing curvilinear boundary crossing probabilities of a general diffusion process on a finite time interval. For broad classes of…
In this paper we propose two behavioral distances that support approximate reasoning on Stochastic Markov Models (SMMs), that are continuous-time stochastic transition systems where the residence time on each state is described by a generic…
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained.…
This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (a.k.a. variable length…
Understanding and predicting how complex systems respond to external perturbations is a central challenge in nonequilibrium statistical physics. Here we consider continuous-time Markov networks, which we subject to perturbations along a…
This paper studies three kinds of long-term behaviours, namely reachability, repeated reachability and persistence, of quantum Markov chains (qMCs). As a stepping-stone, we introduce the notion of bottom strongly connected component (BSCC)…
We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a…
The paper is concerned with approximating the distribution of a sum W of n integer valued random variables Y_i, whose distributions depend on the state of an underlying Markov chain X. The approximation is in terms of a translated Poisson…
Markov chain Monte Carlo (MCMC) methods are foundational algorithms for Bayesian inference and probabilistic modeling. However, most MCMC algorithms are inherently sequential and their time complexity scales linearly with the sequence…
Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…
We give a number of results on approximations of Markov kernels in total variation and Wasserstein norms weighted by a Lyapunov function. The results are applied to examples from Bayesian statistics where approximations to transition…
An algorithm for estimating quasi-stationary distribution of finite state space Markov chains has been proven in a previous paper. Now this paper proves a similar algorithm that works for general state space Markov chains under very general…
The design of provably correct controllers for continuous-state stochastic systems crucially depends on approximate finite-state abstractions and their accuracy quantification. For this quantification, one generally uses approximate…
Herein, the Hidden Markov Model is expanded to allow for Markov chain observations. In particular, the observations are assumed to be a Markov chain whose one step transition probabilities depend upon the hidden Markov chain. An…
We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…