相关论文: Constant conditional entropy and related hypothese…
Coherent information is a useful concept in quantum information theory. It connects with other notions in data processing. In this short remark, we discuss the coherent information saturating its upper bound. A necessary and sufficient…
The analyticity of the entropy and relative entropy rates of continuous-state hidden Markov models is studied here. Using the analytic continuation principle and the stability properties of the optimal filter, the analyticity of these rates…
We investigated long range correlations in two literary texts, Moby Dick by H. Melville and Grimm's tales. The analysis is based on the calculation of entropy like quantities as the mutual information for pairs of letters and the entropy,…
In this short note, I show how a recent result of Alhejji and Smith [arXiv:1909.00787] regarding an optimal uniform continuity bound for classical conditional entropy leads to an optimal uniform continuity bound for quantum conditional…
Recently long range correlations were detected in nucleotide sequences and in human writings by several authors. We undertake here a systematic investigation of two books, Moby Dick by H. Melville and Grimm's tales, with respect to the…
A method for classification of complex time series using coarse-grained entropy rates (CER's) is presented. The CER's, which are computed from information-theoretic functionals -- redundancies, are relative measures of regularity and…
Universal coding of integers~(UCI) is a class of variable-length code, such that the ratio of the expected codeword length to $\max\{1,H(P)\}$ is within a constant factor, where $H(P)$ is the Shannon entropy of the decreasing probability…
A fundamental question in causal inference is whether it is possible to reliably infer manipulation effects from observational data. There are a variety of senses of asymptotic reliability in the statistical literature, among which the most…
This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher…
We study maximum-entropy inference for finite-dimensional quantum states under linear moment constraints. Given expectation values of finitely many observables, the feasible set of states is convex but typically non-unique. The…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…
Recently, two extensions of Wyner's common information\textemdash exact and R\'enyi common informations\textemdash were introduced respectively by Kumar, Li, and El Gamal (KLE), and the present authors. The class of common information…
Inspired by Hilberg's hypothesis, which states that mutual information between blocks for natural language grows like a power law, we seek for links between power-law growth rate of algorithmic mutual information and of some estimator of…
We introduce a new information-theoretic formulation of quantum measurement uncertainty relations, based on the notion of relative entropy between measurement probabilities. In the case of a finite-dimensional system and for any approximate…
This paper presents a unified framework, integrating information theory and statistical mechanics, to connect metric failure in high-dimensional data with emergence in complex systems. We propose the "Information Dilution Theorem,"…
Recent work by Ishibashi, Maeda, and Mefford shows that the validity of the QNEC is sensitive to the IR regulator that one uses to define the entropy and its variations. In this note we discuss two general options that preserve both the…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
We study the problem of entropy calibration, which asks whether a language model's entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with entropy per step increasing as generations…
Partial Information Decomposition (PID) seeks to disentangle how information about a target variable is distributed across multiple sources, separating redundant, unique, and synergistic contributions. Despite extensive theoretical…
The conditional entropy power inequality is a fundamental inequality in information theory, stating that the conditional entropy of the sum of two conditionally independent vector-valued random variables each with an assigned conditional…