Related papers: Model Checking Markov Chains as Distribution Trans…
Statistical model checking (SMC) is a technique for analysis of probabilistic systems that may be (partially) unknown. We present an SMC algorithm for (unbounded) reachability yielding probably approximately correct (PAC) guarantees on the…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
This paper introduces a Markov chain approach that allows a rigorous analysis of agent based opinion dynamics as well as other related agent based models (ABM). By viewing the ABM dynamics as a micro description of the process, we show how…
A statistical test is presented to decide whether data are adequately described by probabilistic functions of finite state Markov chains (''hidden Markov models'') as applied in the analysis of ion channel data. Particularly, the test can…
Design and control of autonomous systems that operate in uncertain or adversarial environments can be facilitated by formal modelling and analysis. Probabilistic model checking is a technique to automatically verify, for a given temporal…
We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov…
We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…
Markov Chains with variable length are useful stochastic models for data compression that avoid the curse of dimensionality faced by that full Markov Chains. In this paper we introduce a Variable Length Markov Chain whose transition…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
We discuss various models for epidemics on networks that rely on Markov chains. Random walks on graphs are often used to predict epidemic spread and to investigate possible control actions to mitigate them. In this study, we demonstrate…
We consider long-time behavior of dynamical systems perturbed by a small noise. Under certain conditions, a slow component of such a motion, which is most important for long- time evolution, can be described as a motion on the cone of…
In this paper we consider Markov chains with transition rates that depend on a small parameter $\varepsilon$. Under a mild assumption on the asymptotics of these transition rates, we describe the behavior of the chain at various…
Metastability is a physical phenomenon ubiquitous in first order phase transitions. A fruitful mathematical way to approach this phenomenon is the study of rare transitions Markov chains. For Metropolis chains associated with Statistical…
We present a data-driven model predictive control scheme for chance-constrained Markovian switching systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…
Flowgraph models provide an alternative approach in modeling a multi-state stochastic process. One of the most widely used stochastic processes that have many real-world applications especially in actuarial models is the Markov jump process…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…
We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…
Computing the stationary distributions of a continuous-time Markov chain (CTMC) involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and the equations cannot be solved…
"Quantum trajectories" are solutions of stochastic differential equations also called Belavkin or Stochastic Schr\"odinger Equations. They describe random phenomena in quantum measurement theory. Two types of such equations are usually…
Discrete time control systems whose dynamics and observations are described by stochastic equations are common in engineering, operations research, health care, and economics. For example, stochastic filtering problems are usually defined…