Related papers: Shannon Entropy for Neutrosophic Information
We show that a proper expression of the uncertainty relation for a pair of canonically-conjugate continuous variables relies on entropy power, a standard notion in Shannon information theory for real-valued signals. The resulting…
A class of estimators of the R\'{e}nyi and Tsallis entropies of an unknown distribution $f$ in $\mathbb{R}^m$ is presented. These estimators are based on the $k$th nearest-neighbor distances computed from a sample of $N$ i.i.d. vectors with…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…
In this work, we derive information-theoretic properties for a modified Tsallis entropy, hereinafter referred to as q-entropy. We introduce the notions of joint q-entropy, conditional q-entropy, relative q-entropy, conditional mutual…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
Entropy is critically examined as a fundamental concept in contemporary science and informatics. Although the typical Shannon entropy provides a proper framework for describing the canonical ensemble, it fails to represent adequately the…
In this paper, we develop the notion of entropy for uniform hypergraphs via tensor theory. We employ the probability distribution of the generalized singular values, calculated from the higher-order singular value decomposition of the…
After Shannon, entropy becomes a fundamental quantity to describe not only uncertainity or chaos of a system but also information carried by the system. Shannon's important discovery is to give a mathematical expression of the mutual…
The uncertainty relation based on the Shannon entropies of the probability densities in position and momentum spaces is improved for quantum systems in arbitrary $D$-dimensional spherically symmetric potentials. To find it, we have used the…
An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic…
We formulate an info-clustering paradigm based on a multivariate information measure, called multivariate mutual information, that naturally extends Shannon's mutual information between two random variables to the multivariate case…
In the present paper we prove a family of tight upper and lower bounds for the Shannon entropy and von Neumann entropy based on the p-norms. This allows us to have an entropy estimate, a criterion for the finiteness of it and a bound on the…
The purpose of this note is to give the general solution of two functional equations connected to the Shannon entropy and also to the Tsallis entropy. As a result of this, we present the regular solution of these equations, as well.…
The form invariance of pseudoadditivity is shown to determine the structure of nonextensive entropies. Nonextensive entropy is defined as the appropriate expectation value of nonextensive information content, similar to the definition of…
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…
Information entropic measures such as Fisher information, Shannon entropy, Onicescu energy and Onicescu Shannon entropy of a symmetric double-well potential are calculated in both position and momentum space. Eigenvalues and eigenvectors of…
This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
Understanding the way in which random entities interact is of key interest in numerous scientific fields. This can range from a full characterization of the joint distribution to single scalar summary statistics. In this work we identify a…