Related papers: Shannon entropy estimation for linear processes
The purpose of this note is to give the general solution of two functional equations connected to the Shannon entropy and also to the Tsallis entropy. As a result of this, we present the regular solution of these equations, as well.…
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…
A method of estimating the joint probability mass function of a pair of discrete random variables is described. This estimator is used to construct the conditional Shannon-R\'eyni-Tsallis entropies estimates. From there almost sure rates of…
We construct in the small-time setting the upper and lower estimates for the transition probability density of a L\'evy process in $\rn$. Our approach relies on the complex analysis technique and the asymptotic analysis of the inverse…
Consider the semiparametric transformation model $\Lambda_{\theta_o}(Y)=m(X)+\epsilon$, where $\theta_o$ is an unknown finite dimensional parameter, the functions $\Lambda_{\theta_o}$ and $m$ are smooth, $\epsilon$ is independent of $X$,…
A simple method for finding the entropy and redundancy of a reasonable long sample of English text by direct computer processing and from first principles according to Shannon theory is presented. As an example, results on the entropy of…
Information flow analysis is a powerful technique for reasoning about the sensitive information exposed by a program during its execution. While past work has proposed information theoretic metrics (e.g., Shannon entropy, min-entropy,…
Entropic measures like Shannon entropy (SE), its quantum mechanical analogue von Neumann entropy, and Kullback-Leibler divergence (KL) are key components in many tools used in physics, information theory, machine learning (ML) and quantum…
This paper introduces an objective metric for evaluating a parsing scheme. It is based on Shannon's original work with letter sequences, which can be extended to part-of-speech tag sequences. It is shown that this regular language is an…
The quality of image encryption is commonly measured by the Shannon entropy over the ciphertext image. However, this measurement does not consider to the randomness of local image blocks and is inappropriate for scrambling based image…
In this short note a method for computing the naive plugin estimator of discrete entropy from a counting Bloom filter will be presented. The method does work reasonably as long as the collision probability in the bloom filter is kept low.
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
We describe an efficient theoretical criterion, suitable for indistinguishable particles to quantify the quantum correlations of any pure two-fermion state, based on the Slater rank concept. It represents the natural generalization of the…
It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried. We replace the English text symbol sequence originally used to illustrate the theory by a discrete,…
A number of inequalities for the weighted entropies is proposed, mirroring properties of a standard (Shannon) entropy and related quantities.
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
We develop a complexity measure for large-scale economic systems based on Shannon's concept of entropy. By adopting Leontief's perspective of the production process as a circular flow, we formulate the process as a Markov chain. Then we…
Stochastic thermodynamics extends classical thermodynamics to small systems in contact with one or more heat baths. It can account for the effects of thermal fluctuations and describe systems far from thermodynamic equilibrium. A basic…
Let $\{X_n: n\in \N\}$ be a linear process with density function $f(x)\in L^2(\R)$. We study wavelet density estimation of $f(x)$. Under some regular conditions on the characteristic function of innovations, we achieve, based on the number…
A physically unclonable function (PUF) is an electronic circuit that produces an intrinsic identifier in response to a challenge. These identifiers depend on uncontrollable variations of the manufacturing process, which make them hard to…