Related papers: Sleeping Beauty and Markov chains

Probabilistic model checking for systems with large or unbounded state space is a challenging computational problem in formal modelling and its applications. Numerical algorithms require an explicit representation of the state space, while…

The embedding problem of Markov transition matrices into continuous-time Markov semigroups is a classic problem that regained a lot of impetus and activities in recent years. We consider it here for the following generalisation of the…

We propose an exact technique to calculate lower bounds of spectral gaps of discrete time reversible Markov chains on finite state sets. Spectral gaps are a common tool for evaluating convergence rates of Markov chains. As an illustration,…

The occurrence of Simpson's paradox (SP) in $2\times 2$ contingency tables has been well studied. The present work comprehensively revisits this problem using a combination of philosophical reflections, causal considerations, and…

Belief Propagation (BP) is a widely used approximation for exact probabilistic inference in graphical models, such as Markov Random Fields (MRFs). In graphs with cycles, however, no exact convergence guarantees for BP are known, in general.…

This paper sets out to resolve how agents ought to act in the Sleeping Beauty problem and various related anthropic (self-locating belief) problems, not through the calculation of anthropic probabilities, but through finding the correct…

In this paper, the question how spiking neural network (SNN) learns and fixes in its internal structures a model of external world dynamics is explored. This question is important for implementation of the model-based reinforcement learning…

Bootstrap is an idea that imposing consistency conditions on a physical system may lead to rigorous and nontrivial statements about its physical observables. In this work, we discuss the bootstrap problem for the invariant measure of the…

This paper investigates a series of optimization problems for one-counter Markov decision processes (MDPs) and integer-weighted MDPs with finite state space. Specifically, it considers problems addressing termination probabilities and…

We study very simple sorting algorithms based on a probabilistic comparator model. In our model, errors in comparing two elements are due to (1) the energy or effort put in the comparison and (2) the difference between the compared…

Given Markov chains and Markov decision processes (MDPs) whose transitions are labelled with non-negative integer costs, we study the computational complexity of deciding whether the probability of paths whose accumulated cost satisfies a…

We study the classical occupancy problem from the viewpoint of its embedding Markov chain. We derive new expressions for the probability mass function and (complementary) distribution function in generalized form. Furthermore, we derive a…

Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…

A new object of the probability theory, two-sided chain of events (symbols), is introduced. A theory of multi-steps Markov chains with long-range memory, proposed earlier in Phys. Rev. E 68, 06117 (2003), is developed and used to establish…

The paper deals with finite-state Markov decision processes (MDPs) with integer weights assigned to each state-action pair. New algorithms are presented to classify end components according to their limiting behavior with respect to the…

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are…

While the standard approach to quantum systems studies length preserving linear transformations of wave functions, the Markov picture focuses on trace preserving operators on the space of Hermitian (self-adjoint) matrices. The Markov…

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…

Switching identities have a long history in potential theory and stochastic analysis. In recent work of Cox and Wang, a switching identity was used to connect an optimal stopping problem and the Skorokhod embedding problem (SEP). Typically…

We consider the generalization of the P\'olya urn scheme with possibly infinite many colors as introduced in \cite{Th-Thesis, BaTH2014, BaTh2016, BaTh2017}. For countable many colors, we prove almost sure convergence of the urn…