Related papers: Entropy estimates of small data sets
Mobile sensor data has been proposed for security-critical applications such as device pairing, proximity detection, and continuous authentication. However, the foundational premise that these signals provide sufficient entropy remains…
We introduce a variant of the R\'enyi entropy definition that aligns it with the well-known H\"older mean: in the new formulation, the r-th order R\'enyi Entropy is the logarithm of the inverse of the r-th order H\"older mean. This brings…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Existing polarization theories have mostly been concerned with Shannon's information measures, such as Shannon entropy and mutual information, and some related measures such as the Bhattacharyya parameter. In this work, we extend…
In this work, we analyze alternative effective sample size (ESS) metrics for importance sampling algorithms, and discuss a possible extended range of applications. We show the relationship between the ESS expressions used in the literature…
We study the estimation of Tsallis entropy of a finite number of independent populations, each following an exponential distribution with the same scale parameter and distinct location parameters for $q>0$. We derive a Stein-type improved…
The paper examines relationships between the Shannon entropy and the $\ell_{\alpha}$-norm for $n$-ary probability vectors, $n \ge 2$. More precisely, we investigate the tight bounds of the $\ell_{\alpha}$-norm with a fixed Shannon entropy,…
Compressed Counting (CC), based on maximally skewed stable random projections, was recently proposed for estimating the p-th frequency moments of data streams. The case p->1 is extremely useful for estimating Shannon entropy of data…
Consider two populations characterized by independent random variables $X_1$ and $X_2$ such that $X_i, i=1,2,$ follows a gamma distribution with an unknown scale parameter $\theta_i>0$, and known shape parameter $\alpha >0$ (the same shape…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
In this study an attempt has been made to propose a way to develop new distribution. For this purpose, we need only idea about distribution function. Some important statistical properties of the new distribution like moments, cumulants,…
We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…
Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using…
Non Performing Asset(NPA) has been in a serious attention by banks over the past few years. NPA cause a huge loss to the banks hence it becomes an extremely critical step in deciding which loans have the capabilities to become an NPA and…
Transfer entropy is a widely used measure for quantifying directed information flows in complex systems. While the challenges of estimating transfer entropy for continuous data are well known, it has two major shortcomings for data of…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
This paper introduces an objective metric for evaluating a parsing scheme. It is based on Shannon's original work with letter sequences, which can be extended to part-of-speech tag sequences. It is shown that this regular language is an…
Measuring incomplete sets of mutually unbiased bases constitutes a sensible approach to the tomography of high-dimensional quantum systems. The unbiased nature of these bases optimizes the uncertainty hypervolume. However, imposing…
Entropy-type integral functionals of densities are widely used in mathematical statistics, information theory, and computer science. Examples include measures of closeness between distributions (e.g., density power divergence) and…
To quantify the complexity of a system, entropy-based methods have received considerable critical attentions in real-world data analysis. Among numerous entropy algorithms, amplitude-based formulas, represented by Sample Entropy, suffer…