相关论文: Non-asymptotic Equipartition Properties for Indepe…
Suppose a string $X_1^n=(X_1,X_2,...,X_n)$ generated by a memoryless source $(X_n)_{n\geq 1}$ with distribution $P$ is to be compressed with distortion no greater than $D\geq 0$, using a memoryless random codebook with distribution $Q$. The…
We prove asymptotic equipartition properties for simple hierarchical structures (modelled as multitype Galton-Watson trees) and networked structures (modelled as randomly coloured random graphs). For example, for large $n$, a networked data…
The Asymptotic Equipartition Property (AEP) in information theory establishes that independent and identically distributed (i.i.d.) states behave in a way that is similar to uniform states. In particular, with appropriate smoothing, for…
The output distribution, when rate is above capacity, is investigated. It is shown that there is an asymptotic equipartition property (AEP) of the typical output sequences, independently of the specific codebook used, as long as the…
We propose a generalization of the asymptotic equipartition property to discrete sources with an ambiguous alphabet, and prove that it holds for irreducible stationary Markov sources with an arbitrary distinguishability relation. Our…
The asymptotic equipartition property (AEP) states that in the limit of a large number of independent and identically distributed (i.i.d.) random experiments, the output sequence is virtually certain to come from the typical set, each…
The classical asymptotic equipartition property is the statement that, in the limit of a large number of identical repetitions of a random experiment, the output sequence is virtually certain to come from the typical set, each member of…
This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error…
We present a development of parts of rate-distortion theory and pattern- matching algorithms for lossy data compression, centered around a lossy version of the Asymptotic Equipartition Property (AEP). This treatment closely parallels the…
New non-asymptotic random coding theorems (with error probability $\epsilon$ and finite block length $n$) based on Gallager parity check ensemble and Shannon random code ensemble with a fixed codeword type are established for discrete input…
This note displays an interesting phenomenon for percentiles of independent but non-identical random variables. Let $X_1,\cdots,X_n$ be independent random variables obeying non-identical continuous distributions and $X^{(1)}\geq \cdots\geq…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
The form invariance of pseudoadditivity is shown to determine the structure of nonextensive entropies. Nonextensive entropy is defined as the appropriate expectation value of nonextensive information content, similar to the definition of…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical R\'enyi…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
The problem of determining the best achievable performance of arbitrary lossless compression algorithms is examined, when correlated side information is available at both the encoder and decoder. For arbitrary source-side information pairs,…
This paper presents a rate-distortion theory for hierarchical networked data structures modelled as tree-indexed multitype process. To be specific, this paper gives a generalized Asymptotic Equipartition Property (AEP) for the Process. The…
Information has an entropic character which can be analyzed within the Statistical Theory in molecular systems. R. Landauer and C.H. Bennett showed that a logical copy can be carried out in the limit of no dissipation if the computation is…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…