Related papers: Sleeping Beauty and Markov chains
We discuss problems posed by the quantitative study of time inhomogeneous Markov chains. The two main notions for our purpose are merging and stability. Merging (also called weak ergodicity) occurs when the chain asymptotically forgets…
A general criterion is given for when a Markov chain trapped with probability p(x) in state x will be almost surely trapped. The quenched (state x is a trap forever with probability p(x)) and annealed (state x traps with probability p(x) on…
For both continuous-time and discrete-time Markov Chains, we provide criteria for inverse problems of classical types of ergodicity: (ordinary) erogodicity, algebraic ergodicity, exponential ergodicity and strong ergodicity. Our criteria…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the…
We study the problem of stopping a Brownian motion at a given distribution $\nu$ while optimizing a reward function that depends on the (possibly randomized) stopping time and the Brownian motion. Our first result establishes that the set…
We present a novel approach to detecting and utilizing symmetries in probabilistic graphical models with two main contributions. First, we present a scalable approach to computing generating sets of permutation groups representing the…
We present a novel approach to detecting and utilizing symmetries in probabilistic graphical models with two main contributions. First, we present a scalable approach to computing generating sets of permutation groups representing the…
A simple model of the new notion of "Markov up" processes is proposed; its positive recurrence and ergodic properties are shown under the appropriate conditions.
We study the multi-armed bandit problem with arms which are Markov chains with rewards. In the finite-horizon setting, the celebrated Gittins indices do not apply, and the exact solution is intractable. We provide approximation algorithms…
Markov Logic Networks (MLN) and Probabilistic Soft Logic (PSL) are widely applied formalisms in Statistical Relational Learning, an emerging area in Artificial Intelligence that is concerned with combining logical and statistical AI.…
The classical and extended occupancy distributions are useful for examining the number of occupied bins in problems involving random allocation of balls to bins. We examine the extended occupancy problem by framing it as a Markov chain and…
Central to model selection is a trade-off between performing a good fit and low model complexity: A model of higher complexity should only be favoured over a simpler model if it provides significantly better fits. In Bayesian terms, this…
Markov-switching models are powerful tools that allow capturing complex patterns from time series data driven by latent states. Recent work has highlighted the benefits of estimating components of these models nonparametrically, enhancing…
B-spline-based hidden Markov models employ B-splines to specify the emission distributions, offering a more flexible modelling approach to data than conventional parametric HMMs. We introduce a Bayesian framework for inference, enabling the…
Learning the undirected graph structure of a Markov network from data is a problem that has received a lot of attention during the last few decades. As a result of the general applicability of the model class, a myriad of methods have been…
In this note we derive the exact probability that a specific state in a two-state Markov chain is visited exactly $k$ times after $N$ transitions. We provide a closed-form solution for $\mathbb{P}(N_l = k \mid N)$, considering initial state…
Markov decision problems (MDPs) provide the foundations for a number of problems of interest to AI researchers studying automated planning and reinforcement learning. In this paper, we summarize results regarding the complexity of solving…
Stochastic flows of an advective-diffusive nature are ubiquitous in physical sciences. Of particular interest is the problem to reconcile observed marginal distributions with a given prior posed by E. Schrodinger in 1932/32 and known as the…
In this work we define a stochastic adding machine associated to the Fibonacci base and to a probabilities sequence $\overline{p}=(p_i)_{i\geq 1}$. We obtain a Markov chain whose states are the set of nonnegative integers. We study…