Related papers: On a Class of Markov Order Estimators Based on PPM…
In this paper we investigate a quantity called conditional entropy of ordinal patterns, akin to the permutation entropy. The conditional entropy of ordinal patterns describes the average diversity of the ordinal patterns succeeding a given…
In this paper we investigate a quantity called conditional entropy of ordinal patterns, akin to the permutation entropy. The conditional entropy of ordinal patterns describes the average diversity of the ordinal patterns succeeding a given…
In adaptive systems, predictors are used to anticipate changes in the systems state or behavior that may require system adaption, e.g., changing its configuration or adjusting resource allocation. Therefore, the quality of predictors is…
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk.…
We provide a new algorithm for solving Risk Sensitive Partially Observable Markov Decisions Processes, when the risk is modeled by a utility function, and both the state space and the space of observations is finite. This algorithm is based…
We introduce a class of numerical schemes for optimal control problems based on a novel Markov chain approximation, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauss-Hermite…
Ergodic optimization aims to describe dynamically invariant probability measures that maximize the integral of a given function. For a wide class of intrinsically ergodic subshifts over a finite alphabet, we show that the space of…
The use of higher-order stochastic processes such as nonlinear Markov chains or vertex-reinforced random walks is significantly growing in recent years as they are much better at modeling high dimensional data and nonlinear dynamics in…
We study the error introduced by entropy regularization in infinite-horizon discrete discounted Markov decision processes. We show that this error decreases exponentially in the inverse regularization strength, both in a weighted…
Finding the correct encoding for a generic dynamical system's trajectory is a complicated task: the symbolic sequence needs to preserve the invariant properties from the system's trajectory. In theory, the solution to this problem is found…
Discrete time random dynamical systems with countably many maps which admit countable Markov partitions on complete metric spaces such that the resulting Markov systems are uniform continuous and contractive are considered. A notion of a…
In this paper, we examine the Renyi entropy rate of stationary ergodic processes. For a special class of stationary ergodic processes, we prove that the Renyi entropy rate always exists and can be polynomially approximated by its defining…
Information theoretic quantities play an important role in various settings in machine learning, including causality testing, structure inference in graphical models, time-series problems, feature selection as well as in providing privacy…
We show how to construct a topological Markov map of the interval whose invariant probability measure is the stationary law of a given stochastic chain of infinite order. In particular we caracterize the maps corresponding to stochastic…
Variable Length Memory Chains (VLMC), which are generalizations of finite order Markov chains, turn out to be an essential tool to modelize random sequences in many domains, as well as an interesting object in contemporary probability…
We study the relation between different Markov Decision Process (MDP) frameworks in the machine learning and econometrics literatures, including the standard MDP, the entropy and general regularized MDP, and stochastic MDP, where the latter…
Often one has a preference order among the different systems that satisfy a given specification. Under a probabilistic assumption about the possible inputs, such a preference order is naturally expressed by a weighted automaton, which…
Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on $S$ elements with independent samples, [Paninski2004] showed…
The goal of this paper is to develop an estimate for the entropy of random long-range correlated symbolic sequences with elements belonging to a finite alphabet. As a plausible model, we use the high-order additive stationary ergodic Markov…
Graded posets frequently arise throughout combinatorics, where it is natural to try to count the number of elements of a fixed rank. These counting problems are often $\#\textbf{P}$-complete, so we consider approximation algorithms for…