Related papers: Mean dimension and rate-distortion function revisi…
This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal…
Direct evaluation of the rate-distortion function has rarely been achieved when it is strictly greater than its Shannon lower bound. In this paper, we consider the rate-distortion function for the distortion measure defined by an…
Let $\{S_i\}_{i=1}^\ell$ be an iterated function system (IFS) on $\R^d$ with attractor $K$. Let $(\Sigma,\sigma)$ denote the one-sided full shift over the alphabet $\{1,..., \ell\}$. We define the projection entropy function $h_\pi$ on the…
Rate-distortion theory provides bounds for compressing data produced by an information source to a specified encoding rate that is strictly less than the source's entropy. This necessarily entails some loss, or distortion, between the…
In this manuscript we show that the metric mean dimension of a free semigroup action satisfies three variational principles: (a) the first one is based on a definition of Shapira's entropy, introduced in \cite{SH} for a singles dynamics and…
Understanding generalization in modern machine learning settings has been one of the major challenges in statistical learning theory. In this context, recent years have witnessed the development of various generalization bounds suggesting…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
We show that a certain type of quasi finite, conservative, ergodic, measure preserving transformation always has a maximal zero entropy factor, generated by predictable sets. We also construct a conservative, ergodic, measure preserving…
It is well-known that $\epsilon$-stable sets have a deep connection with the topological entropy of dynamical systems. In the present paper, we investigate the relationships of three types of upper metric mean dimensions with potential…
We investigate lossy compression (source coding) of data in the form of permutations. This problem has direct applications in the storage of ordinal data or rankings, and in the analysis of sorting algorithms. We analyze the rate-distortion…
The manifold of empirical mean values of statistical data ad infinitum has a geometric shape that depends on the probability measure that governs the generating model. Large deviation theory produces entropy functions that depend on both…
Using fish-covering model, this paper intuitively explains how to extend Hartley's information formula to the generalized information formula step by step for measuring subjective information: metrical information (such as conveyed by…
We determine the extent to which certain classes of fractionally `smooth' continuous mappings between metric spaces distort various dimensions, including the Hausdorff, upper Minkowski (box-counting), and upper intermediate dimensions. Our…
The matrix-based R\'enyi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This intriguing property makes it widely applied in statistical…
Motivated by questions in lossy data compression and by theoretical considerations, we examine the problem of estimating the rate-distortion function of an unknown (not necessarily discrete-valued) source from empirical data. Our focus is…
Porosity and dimension are two useful, but different, concepts that quantify the size of fractal sets and measures. An active area of research concerns understanding the relationship between these two concepts. In this article we will…
We propose a "decomposition method" to prove non-asymptotic bound for the convergence of empirical measures in various dual norms. The main point is to show that if one measures convergence in duality with sufficiently regular observables,…
We characterize the rate-distortion function for zero-mean stationary Gaussian sources under the MSE fidelity criterion and subject to the additional constraint that the distortion is uncorrelated to the input. The solution is given by two…
We introduce a new information-theoretic formulation of quantum measurement uncertainty relations, based on the notion of relative entropy between measurement probabilities. In the case of a finite-dimensional system and for any approximate…
Permutation entropy measures the complexity of deterministic time series via a data symbolic quantization consisting of rank vectors called ordinal patterns or just permutations. The reasons for the increasing popularity of this entropy in…