Related papers: The embedding problem for Markov matrices
Markov decision processes (MDP) are a well-established model for sequential decision-making in the presence of probabilities. In robust MDP (RMDP), every action is associated with an uncertainty set of probability distributions, modelling…
This paper considers large families of Markov chains (MCs) that are defined over a set of parameters with finite discrete domains. Such families occur in software product lines, planning under partial observability, and sketching of…
In the context of Markov decision processes running in continuous time, one of the most intriguing challenges is the efficient approximation of finite horizon reachability objectives. A multitude of sophisticated model checking algorithms…
This paper consists of four parts. In the first part, we explain what eigenvalues we are interested in and show the difficulties of the study on the first (non-trivial) eigenvalue through examples. In the second part, we present some (dual)…
A branching process in a Markovian environment consists of an irreducible Markov chain on a set of "environments" together with an offspring distribution for each environment. At each time step the chain transitions to a new random…
The paper presents efficient approaches for evaluating convergence rate in total variation for finite and general linear Markov chains. The motivation for studying convergence rate in this metric is its usefulness in various limit theorems.…
We calculate exact convergence times to reach random bipartite entanglement for various random protocols. The eigenproblem of a Markovian chain governing the process is mapped to a spin chain, thereby obtaining exact expression for the gap…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
Markovian credit migration models are a reasonably standard tool nowadays, but there are fundamental difficulties with calibrating them. We show how these are resolved using a simplified form of matrix generator and explain why risk-neutral…
We give Hoeffding and Bernstein-type concentration inequalities for the largest eigenvalue of sums of random matrices arising from a Markov chain. We consider time-dependent matrix-valued functions on a general state space, generalizing…
Understanding the generative mechanism of a natural system is a vital component of the scientific method. Here, we investigate one of the fundamental steps toward this goal by presenting the minimal generator of an arbitrary binary Markov…
The limited connectivity of current and next-generation quantum annealers motivates the need for efficient graph-minor embedding methods. These methods allow non-native problems to be adapted to the target annealer's architecture. The…
In this paper, we study the problem of approximately computing the product of two real matrices. In particular, we analyze a dimensionality-reduction-based approximation algorithm due to Sarlos [1], introducing the notion of nuclear rank as…
We study ergodic properties of a class of Markov-modulated general birth-death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that…
RNA motifs typically consist of short, modular patterns that include base pairs formed within and between modules. Estimating the abundance of these patterns is of fundamental importance for assessing the statistical significance of matches…
This paper considers the problem of finding strategies that satisfy a mixture of sure and threshold objectives in Markov decision processes. We focus on a single $\omega$-regular objective expressed as parity that must be surely met while…
In this article, we extend the study of embedded corrector problems, that we have previously introduced in the context of the homogenization of scalar diffusive equations, to the context of homogenized elastic properties of materials. This…
We study a simple embedding technique based on a matrix of personalized PageRank vectors seeded on a random set of nodes. We show that the embedding produced by the element-wise logarithm of this matrix (1) are related to the spectral…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
Many learning algorithms can be represented as Markov processes, and understanding their generalization error is a central topic in learning theory. For specific continuous-time noisy algorithms, a prominent analysis technique relies on…