Related papers: A class of R\'{e}nyi information estimators for mu…
We study statistical/computational tradeoffs for the following density estimation problem: given $k$ distributions $v_1, \ldots, v_k$ over a discrete domain of size $n$, and sampling access to a distribution $p$, identify $v_i$ that is…
We provide finite-sample analysis of a general framework for using k-nearest neighbor statistics to estimate functionals of a nonparametric continuous probability density, including entropies and divergences. Rather than plugging a…
We study density estimation for classes of shift-invariant distributions over $\mathbb{R}^d$. A multidimensional distribution is "shift-invariant" if, roughly speaking, it is close in total variation distance to a small shift of it in any…
In a previous article, a least square regression estimation procedure was proposed: first, we condiser a family of functions and study the properties of an estimator in every unidimensionnal model defined by one of these functions; we then…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
This paper proposes a new probabilistic classification algorithm using a Markov random field approach. The joint distribution of class labels is explicitly modelled using the distances between feature vectors. Intuitively, a class label…
The Sharma-Mittal entropies generalize the celebrated Shannon, R\'enyi and Tsallis entropies. We report a closed-form formula for the Sharma-Mittal entropies and relative entropies for arbitrary exponential family distributions. We…
The formalism of statistical mechanics can be generalized by starting from more general measures of information than the Shannon entropy and maximizing those subject to suitable constraints. We discuss some of the most important examples of…
Calculating the Shannon entropy for symbolic sequences has been widely considered in many fields. For descriptive statistical problems such as estimating the N-gram entropy of English language text, a common approach is to use as much data…
The problem of accurate nonparametric estimation of distributional functionals (integral functionals of one or more probability distributions) has received recent interest due to their wide applicability in signal processing, information…
It is well known that to estimate the Shannon entropy for symbolic sequences accurately requires a large number of samples. When some aspects of the data are known it is plausible to attempt to use this to more efficiently compute entropy.…
We revisit extending the Kolmogorov-Smirnov distance between probability distributions to the multidimensional setting and make new arguments about the proper way to approach this generalization. Our proposed formulation maximizes the…
We show that different entropic measures of fluctuations lead to contradictory uncertainty relations for two complementary observables. We apply Tsallis and R\'{e}nyi entropies to the joint distribution emerging from a noisy simultaneous…
More and more works deal with statistical systems far from equilibrium, dominated by unidirectional stochastic processes augmented by rare resets. We analyze the construction of the entropic distance measure appropriate for such dynamics.…
In this paper, we study entropic uncertainty relations on a finite-dimensional Hilbert space and provide several tighter bounds for multi-measurements, with some of them also valid for R\'{e}nyi and Tsallis entropies besides the Shannon…
Entanglement criteria for an $n$-partite quantum system with continuous variables are formulated in terms of R\'{e}nyi entropies. R\'{e}nyi entropies are widely used as a good information measure due to many nice properties. Derived…
An analytical expression of probability density function (PDF) of velocity fluctuation is derived with the help of the statistics based on generalized entropy (the Tsallis entropy or the R\'{e}nyi entropy). It is revealed that the derived…
The Tsallis and R\'enyi entropies are important quantities in the information theory, statistics and related fields because the Tsallis entropy is an one parameter generalization of the Shannon entropy and the R\'enyi entropy includes…
The recently developed matrix based Renyi's entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the…
This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized…