Related papers: Markov Chains with Maximum Return Time Entropy for…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…
Motivated by the Internet-of-things and sensor networks for cyberphysical systems, the problem of dynamic sensor activation for the tracking of a time-varying process is examined. The tradeoff is between energy efficiency, which decreases…
Reversibility is a key property of Markov chains, central to algorithms such as Metropolis-Hastings and other MCMC methods. Yet many applications yield non-reversible chains, motivating the problem of approximating them by reversible ones…
Attention mechanisms have been extensively employed in various applications, including time series modeling, owing to their capacity to capture intricate dependencies; however, their utility is often constrained by quadratic computational…
We consider the problem of sampling from the uniform distribution on the set of Eulerian orientations of subgraphs of the triangular lattice. Although it is known that this can be achieved in polynomial time for any graph, the algorithm…
A rigorous and largely self-contained account of (a) the bread-and-butter concepts and techniques in Markov chain theory and (b) the long-term behaviour of chains. As much as possible, the treatment is probabilistic instead of analytical (I…
This work derives a theoretical value for the entropy of a Linear Additive Markov Process (LAMP), an expressive model able to generate sequences with a given autocorrelation structure. While a first-order Markov Chain model generates new…
We propose a new model for formalizing reward collection problems on graphs with dynamically generated rewards which may appear and disappear based on a stochastic model. The *robot routing problem* is modeled as a graph whose nodes are…
It has become increasingly easy nowadays to collect approximate posterior samples via fast algorithms such as variational Bayes, but concerns exist about the estimation accuracy. It is tempting to build solutions that exploit approximate…
The inference of thermodynamic quantities from the description of an only partially accessible physical system is a central challenge in stochastic thermodynamics. A common approach is coarse-graining, which maps the dynamics of such a…
This paper addresses the problem of finding a minimum-cost $m$-state Markov chain $(S_0,\ldots,S_{m-1})$ in a large set of chains. The chains studied have a reward associated with each state. The cost of a chain is its "gain", i.e., its…
The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…
We investigate the mixing properties of a model of reversible Markov chains in random environment, which notably contains the simple random walk on the superposition of a deterministic graph and a second graph whose vertex set has been…
In this work, we study the real-time tracking and reconstruction of an information source with the purpose of actuation. A device monitors the state of the information source and transmits status updates to a receiver over a wireless…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…
We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…
Discrete-time discrete-state finite Markov chains are versatile mathematical models for a wide range of real-life stochastic processes. One of most common tasks in studies of Markov chains is computation of the stationary distribution.…
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…