Related papers: Are Parametric Markov Chains Monotonic?
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…
This paper studies parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a finite set of parameters. Fixing values for all parameters…
We provide a sufficient criterion for the unique parameter identification of combinatorially symmetric Hidden Markov Models based on the structure of their transition matrix. If the observed states of the chain form a zero forcing set of…
We consider the problem of computing the satisfaction probability of a formula for stochastic models with parametric uncertainty. We show that this satisfaction probability is a smooth function of the model parameters. This enables us to…
Stochastically monotone Markov chains arise in many applied domains, especially in the setting of queues and storage systems. Poisson's equation is a key tool for analyzing additive functionals of such models, such as cumulative sums of…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
Statistical model checking (SMC) is a technique for analysis of probabilistic systems that may be (partially) unknown. We present an SMC algorithm for (unbounded) reachability yielding probably approximately correct (PAC) guarantees on the…
The preparation of the stationary distribution of irreducible, time-reversible Markov chains is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of…
We consider the computational task of sampling a bit string $x$ from a distribution $\pi(x)=|\langle x|\psi\rangle|^2$, where $\psi$ is the unique ground state of a local Hamiltonian $H$. Our main result describes a direct link between the…
We analyze the notions of monotonicity and complete monotonicity for Markov Chains in continuous-time, taking values in a finite partially ordered set. Similarly to what happens in discrete-time, the two notions are not equivalent. However,…
This paper is concerned with the solution of the optimal stopping problem associated to the valuation of Perpetual American options driven by continuous time Markov chains. We introduce a new dynamic approach for the numerical pricing of…
This article surveys recent advancements of strategy designs for persistent robotic surveillance tasks with the focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way where…
We investigate multivariate regular variation in the context of time-homogeneous Markov chains on general vector spaces and in random coefficient linear models. In the first part, we show that the regular variation of the stationary…
Unambiguous automata are nondeterministic automata in which every word has at most one accepting run. In this paper we give a polynomial-time algorithm for model checking discrete-time Markov chains against \omega-regular specifications…
Autonomous systems with machine learning-based perception can exhibit unpredictable behaviors that are difficult to quantify, let alone verify. Such behaviors are convenient to capture in probabilistic models, but probabilistic model…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
We introduce a new embarrassingly parallel parameter learning algorithm for Markov random fields with untied parameters which is efficient for a large class of practical models. Our algorithm parallelizes naturally over cliques and, for…
Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…
For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…
Motivated by techniques developed in recent progress on lower bounds for sublinear time algorithms (Behnezhad, Roghani and Rubinstein, STOC 2023, FOCS 2023, and STOC 2024) we introduce and study a new class of randomized algorithmic…