相关论文: Constant conditional entropy and related hypothese…

We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…

In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the…

Human language has a distinct systematic structure, where utterances break into individually meaningful words which are combined to form phrases. We show that natural-language-like systematicity arises in codes that are constrained by a…

The logarithm of the number of Eulerian orientations, normalised by the number of vertices, is known as the residual entropy in studies of ice-type models on graphs. The spanning tree entropy depends similarly on the number of spanning…

An information theory model is used to construct a molecular explanation why hydrophobic solvation entropies measured in calorimetry of protein unfolding converge at a common temperature. The entropy convergence follows from the weak…

The Principle of Maximum Entropy is a rigorous technique for estimating an unknown distribution given partial information while simultaneously minimizing bias. However, an important requirement for applying the principle is that the…

Information (I) is defined as the amount of the data after data compression. The first law of information theory: the total amount of data L (the sum of entropy S and information I) of an isolated system remains unchanged. The second law of…

We present two novel models of document coherence and their application to information retrieval (IR). Both models approximate document coherence using discourse entities, e.g. the subject or object of a sentence. Our first model views text…

We approach the physical implications of the non-commutative nature of Complementary Observable Algebras (COA) from an information theoretic perspective. In particular, we derive a general \textit{entropic certainty principle} stating that…

Statistical modeling of physical laws connects experiments with mathematical descriptions of natural phenomena. The modeling is based on the probability density of measured variables expressed by experimental data via a kernel estimator. As…

We show that a proper expression of the uncertainty relation for a pair of canonically-conjugate continuous variables relies on entropy power, a standard notion in Shannon information theory for real-valued signals. The resulting…

We revisit the maximum-entropy inference of the state of a finite-level quantum system under linear constraints. The constraints are specified by the expected values of a set of fixed observables. We point out the existence of…

Maximal repetition of a string is the maximal length of a repeated substring. This paper investigates maximal repetition of strings drawn from stochastic processes. Strengthening previous results, two new bounds for the almost sure growth…

Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…

Second order asymptotics of fixed-length source coding and intrinsic randomness is discussed with a constant error constraint. There was a difference between optimal rates of fixed-length source coding and intrinsic randomness, which never…

We present a bouquet of continuity bounds for quantum entropies, falling broadly into two classes: First, a tight analysis of the Alicki-Fannes continuity bounds for the conditional von Neumann entropy, reaching almost the best possible…

We introduce an axiomatic approach for characterizing quantum conditional entropy. Our approach relies on two physically motivated axioms: monotonicity under conditional majorization and additivity. We show that these two axioms provide…

This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large…

The paper concerns the rates of power-law growth of mutual information computed for a stationary measure or for a universal code. The rates are called Hilberg exponents and four such quantities are defined for each measure and each code:…

We show that Frenkel's integral representation of the quantum relative entropy provides a natural framework to derive continuity bounds for quantum information measures. Our main general result is a dimension-independent semi-continuity…