Related papers: Shannon entropy estimation for linear processes
Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…
This paper studies the Shannon regime for the random displacement of stationary point processes. Let each point of some initial stationary point process in $\R^n$ give rise to one daughter point, the location of which is obtained by adding…
This paper deals with the kernel density estimator based on the so-called sinc (or Fourier integral) kernel $K(x)=(\pi x)^{-1}\sin x$. We study in detail both asymptotic and finite sample properties of this estimator. It is shown that,…
The relative entropy for two different degenerate diffusion processes is estimated by using the Wasserstein distance of initial distributions and the difference between coefficients. As applications, the entropy cost inequality and…
From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of…
We provide a new way of constraining the relative scintillation efficiency L_eff for liquid xenon. Using a simple estimate for the electronic and nuclear stopping powers together with an analysis of recombination processes we predict both…
A plug-in estimator of entropy is the entropy of the distribution where probabilities of symbols or blocks have been replaced with their relative frequencies in the sample. Consistency and asymptotic unbiasedness of the plug-in estimator…
Shannon's Entropy Power Inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key…
We prove a Bernstein-type bound for the difference between the average of negative log-likelihoods of independent discrete random variables and the Shannon entropy, both defined on a countably infinite alphabet. The result holds for the…
In this paper, we develop the notion of entropy for uniform hypergraphs via tensor theory. We employ the probability distribution of the generalized singular values, calculated from the higher-order singular value decomposition of the…
The Renyi entropy is a generalisation of the Shannon entropy that is sensitive to the fine details of a probability distribution. We present results for the Renyi entropy of the totally asymmetric exclusion process (TASEP). We calculate…
Fisher information, Shannon information entropy and Statistical Complexity are calculated for the interface of a normal metal and a superconductor, as a function of the temperature for several materials. The order parameter $\Psi({\bf r})$…
The $\epsilon$-machine is a stochastic process' optimal model -- maximally predictive and minimal in size. It often happens that to optimally predict even simply-defined processes, probabilistic models -- including the $\epsilon$-machine --…
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear…
In this article we introduce an entropy-based, scale-dependent centrality that is evaluated as the Shannon entropy of the distribution at time t of a continuous-time random walk. It ranks nodes as a function of the time t, which acts as a…
We establish a discrete analog of the R\'enyi entropy comparison due to Bobkov and Madiman. For log-concave variables on the integers, the min entropy is within log e of the usual Shannon entropy. Additionally we investigate the entropic…
We present the Shannon entropy as an indicator of spatial resolution for morphology of resonance mode pattern in dielectric micro cavity. We obtain two types of optimized mesh point for the minimum and maximum sizes, respectively. The…
In this work we determine and discuss the entropic uncertainty measures of Shannon type for all the discrete stationary states of the multidimensional harmonic systems directly in terms of the states' hyperquantum numbers, the…
Entropy measures are effective features for time series classification problems. Traditional entropy measures, such as Shannon entropy, use probability distribution function. However, for the effective separation of time series, new entropy…