Related papers: Predictive Complexity and Generalized Entropy Rate…
An instability property of the Birkhoff's ergodic theorem and related asymptotic laws with respect to small violations of algorithmic randomness is studied. The Shannon--McMillan--Breiman theorem and all universal compression schemes are…
The idea of a parsing of a stationary process according to a collection of words is introduced, and the basic framework required for the asymptotic analysis of these parsings is presented. We demonstrate how the pointwise ergodic theorem…
The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and…
The Shannon-McMillan-Breiman theorem is one of the most important results in information theory, which can describe the random ergodic process, and its proof uses the famous Birkhoff ergodic theorem, so it can be seen that it plays a…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
We study two-player (zero-sum) concurrent mean-payoff games played on a finite-state graph. We focus on the important sub-class of ergodic games where all states are visited infinitely often with probability 1. The algorithmic study of…
Predictive inference requires balancing statistical accuracy against informational complexity, yet the choice of complexity measure is usually imposed rather than derived. We treat econometric objects as predictive rules, mappings from…
It is known that a gambler repeating a game with positive expected value has a positive probability to never go broke. We use the mass transport method to prove the generalization of this fact where the gains from the bets form a…
The theorem of Shannon-McMillan-Breiman states that for every generating partition on an ergodic system, the exponential decay rate of the measure of cylinder sets equals the metric entropy almost everywhere (provided the entropy is…
Four estimators of the directed information rate between a pair of jointly stationary ergodic finite-alphabet processes are proposed, based on universal probability assignments. The first one is a Shannon--McMillan--Breiman type estimator,…
We introduce the notion of common conditional expectation to investigate Birkhoff's ergodic theorem and subadditive ergodic theorem for invariant upper probabilities. If in addition, the upper probability is ergodic, we construct an…
In this paper, we consider contention resolution algorithms that are augmented with predictions about the network. We begin by studying the natural setup in which the algorithm is provided a distribution defined over the possible network…
In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…
We develop value iteration-based algorithms to solve in a unified manner different classes of combinatorial zero-sum games with mean-payoff type rewards. These algorithms rely on an oracle, evaluating the dynamic programming operator up to…
We study the relations between the recently proposed machine-independent quantum complexity of P. Gacs~\cite{Gacs} and the entropy of classical and quantum systems. On one hand, by restricting Gacs complexity to ergodic classical dynamical…
We describe and develop a close relationship between two problems that have customarily been regarded as distinct: that of maximizing entropy, and that of minimizing worst-case expected loss. Using a formulation grounded in the equilibrium…
The ergodic hypothesis is examined for energetically open fluid systems represented by the barotropic Navier--Stokes equations with general inflow/outflow boundary conditions. We show that any globally bounded trajectory generates a…
We introduce a new isomorphism-invariant notion of entropy for measure preserving actions of arbitrary countable groups on probability spaces, which we call orbital Rokhlin entropy. It employs Danilenko's orbital approach to entropy of a…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
We study zero-sum games in the space of probability distributions over the Euclidean space $\mathbb{R}^d$ with entropy regularization, in the setting when the interaction function between the players is smooth and strongly convex-strongly…