Related papers: Unbiased Estimators for Entropy and Class Number
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
This paper studies nonparametric empirical Bayes methods in a heterogeneous parameters framework that features unknown means and variances. We provide extended Tweedie's formulae that express the (infeasible) optimal estimators of…
We examine the recently introduced NSB estimator of entropies of severely undersampled discrete variables and devise a procedure for calculating the involved integrals. We discover that the output of the estimator has a well defined limit…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
The paper suggests the concepts of an upper entropy and a lower entropy. We propose a new axiomatic definition, namely, upper entropy axioms, inspired by axioms of metric spaces, and also formulate lower entropy axioms. We also develop weak…
We derive mean-unbiased estimators for the structural parameter in instrumental variables models with a single endogenous regressor where the sign of one or more first stage coefficients is known. In the case with a single instrument, there…
Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using…
Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…
Assuming squared error loss, we show that finding unbiased estimators and Bayes estimators can be treated as using a pair of linear operators that operate between two Hilbert spaces. We note that these integral operators are adjoint and…
We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…
We provide a general method to analyze the asymptotic properties of a variety of estimators of continuous time diffusion processes when the data are not only discretely sampled in time but the time separating successive observations may…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
In this paper, we explicitly derive unbiased estimators for various functions of the rate parameter of the exponential distribution in the absence of a location parameter, including powers of the rate parameter, the $q$th quantile, the…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
We show how universal codes can be used for solving some of the most important statistical problems for time series. By definition, a universal code (or a universal lossless data compressor) can compress any sequence generated by a…
We study the computational complexity and variance of multilevel best linear unbiased estimators introduced in [D. Schaden and E. Ullmann, SIAM/ASA J. Uncert. Quantif., (2020)]. We specialize the results in this work to PDE-based models…
According to a previous conjecture, spatial and temporal Lyapunov exponents of chaotic extended systems can be obtained from derivatives of a suitable function: the entropy potential. The validity and the consequences of this hypothesis are…
We design, implement and test a simple algorithm which computes the approximate entropy of a finite binary string of arbitrary length. The algorithm uses a weighted average of the Shannon Entropies of the string and all but the last binary…
We present a numerical study of the application of the Shannon entropy technique to the planar restricted three-body problem in the vicinity of first-order interior mean-motion resonances with the perturber. We estimate the diffusion…
The paper presents an extension of Shannon entropy for neutrosophic information. This extension uses a new formula for distance between two neutrosophic triplets. In addition, the obtained results are particularized for bifuzzy,…