Related papers: The Entropy of Conditional Markov Trajectories

In this paper, we consider the problem of controlling a partially observed Markov decision process (POMDP) in order to actively estimate its state trajectory over a fixed horizon with minimal uncertainty. We pose a novel active smoothing…

Let $(\xi_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and…

Motivated by a model presented by S. Gudder, we study a quantum generalization of Markov chains and discuss the relation between these maps and open quantum random walks, a class of quantum channels described by S. Attal et al. We consider…

Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…

The correspondence between the cross-entropy method and the zero-variance approximation to simulate a rare event problem in Markov chains is shown. This leads to a sufficient condition that the cross-entropy estimator is asymptotically…

Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…

This paper considers the speed of convergence (mixing) of a finite Markov kernel $P$ with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines either a discrete-time Markov chain (with the $n$-step…

For a Markov chain $Y$ with values in a Polish space, consider the entrance chain, obtained by sampling $Y$ at the moments when it enters a fixed set $A$ from its complement $A^c$. Similarly, consider the exit chain, obtained by sampling…

We present a principled approach for estimating the matrix of microscopic rates among states of a Markov process, given only its stationary state population distribution and a single average global kinetic observable. We adapt Maximum…

We present a method to sample Markov-chain trajectories constrained to both the initial and final conditions, which we term Markov bridges. The trajectories are conditioned to end in a specific state at a given time. We derive the master…

In this paper a method based on a Markov chain Monte Carlo (MCMC) algorithm is proposed to compute the probability of a rare event. The conditional distribution of the underlying process given that the rare event occurs has the probability…

A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…

The main goal of this paper is to develop an estimate for the entropy of random stationary ergodic symbolic sequences with elements belonging to a finite alphabet. We present here the detailed analytical study of the entropy for the…

This work derives a theoretical value for the entropy of a Linear Additive Markov Process (LAMP), an expressive model able to generate sequences with a given autocorrelation structure. While a first-order Markov Chain model generates new…

For a Markovian dynamics on discrete states, the logarithmic ratio of waiting-time distributions between two successive, instantaneous transitions in forward and backward direction is a measure of time-irreversibility. It thus serves as an…

Let {X_n,n\geq0} be a Markov chain on a general state space X with transition probability P and stationary probability \pi. Suppose an additive component S_n takes values in the real line R and is adjoined to the chain such that…

Stochastic processes under resetting at random times have attracted a lot of attention in recent years and served as illustrations of nontrivial and interesting static and dynamic features of stochastic dynamics. In this paper, we aim to…

Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…

Filyokov and Karpov [Inzhenerno-Fizicheskii Zhurnal 13, 624 (1967)] have proposed a theory of non-equilibrium steady states in direct analogy with the theory of equilibrium states : the principle is to maximize the Shannon entropy…

Any continuous curve in a higher dimensional space can be considered a trajectory that can be parameterized by a single variable, usually taken as time. It is well known that a continuous curve can have a fractional dimensionality, which…