Related papers: Statistical tests based on Renyi entropy estimatio…
A generalization of Gauss's principle is used to derive the error laws corresponding to Types II and VII distributions in Pearson's classification scheme. Student's $r$-pdf (Type II) governs the distribution of the internal energy of a…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
Testing for the equality of two high-dimensional distributions is a challenging problem, and this becomes even more challenging when the sample size is small. Over the last few decades, several graph-based two-sample tests have been…
The recently developed matrix based Renyi's entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
Distributions of abundances or frequencies play an important role in many fields of science, from biology to sociology, as does the R\'enyi entropy, which measures the diversity of a statistical ensemble. We derive a mathematical relation…
Nonparametric tests for equality of multivariate distributions are frequently desired in research. It is commonly required that test-procedures based on relatively small samples of vectors accurately control the corresponding Type I Error…
In a recent paper, the authors proposed a general methodology for probabilistic learning on manifolds. The method was used to generate numerical samples that are statistically consistent with an existing dataset construed as a realization…
In this paper, the statistical properties of Newton s method algorithm output in a specific case have been studied. The relative frequency density of this sample converges to a well-defined function, prompting us to explore its…
The Levy-type distributions are derived using the principle of maximum Tsallis nonextensive entropy both in the full and half spaces. The rates of convergence to the exact Levy stable distributions are determined by taking the N-fold…
The method of optimizing entropy is used to (i) conduct Asymptotic Hypothesis Testing and (ii) determine the particle distribution for which Entropy is maximized. This paper focuses on two related applications of Information Theory:…
We conclude a sequence of work by giving near-optimal sketching and streaming algorithms for estimating Shannon entropy in the most general streaming model, with arbitrary insertions and deletions. This improves on prior results that obtain…
Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…
Two new goodness of fit tests for the Pareto type-I distribution for complete and right censored data are proposed using fixed point characterization based on Steins type identity. The asymptotic distributions of the test statistics under…
The Shannon entropy is a widely used summary statistic, for example, network traffic measurement, anomaly detection, neural computations, spike trains, etc. This study focuses on estimating Shannon entropy of data streams. It is known that…
Recent improvements in the method of estimating Renyi entropies from measurements of coincidences between the events observed in high energy collisions are reviewed. A new, more precise, formulation of the method is presented and its…
The Boltzmann-Gibbs celebrated entropy $S_{BG}=-k\sum_ip_i \ln p_i$ is {\it concave} (with regard to all probability distributions $\{p_i\}$) and {\it stable} (under arbitrarily small deformations of any given probability distribution). It…
A general and relatively simple method for construction of multivariate goodness-of-fit tests is introduced. The proposed test is applied to elliptical distributions. The method is based on a characterization of probability distributions…
In this paper, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of…
Two families of dependence measures between random variables are introduced. They are based on the R\'enyi divergence of order $\alpha$ and the relative $\alpha$-entropy, respectively, and both dependence measures reduce to Shannon's mutual…