Related papers: Statistical Inference for R\'enyi Entropy Function…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
We prove the large deviation principle for several entropy and cross entropy estimators based on return times and waiting times on shift spaces over finite alphabets. We consider shift-invariant probability measures satisfying some…
This paper introduces a class of k-nearest neighbor ($k$-NN) estimators called bipartite plug-in (BPI) estimators for estimating integrals of non-linear functions of a probability density, such as Shannon entropy and R\'enyi entropy. The…
We investigate quantum R\'enyi entropic quantities, specifically those derived from 'sandwiched' divergence. This divergence is one of several proposed R\'enyi generalisations of the quantum relative entropy. We may define R\'enyi…
We find the value of constants related to constraints in characterization of some known statistical distributions and then we proceed to use the idea behind maximum entropy principle to derive generalized version of this distributions using…
Distribution function is essential in statistical inference, and connected with samples to form a directed closed loop by the correspondence theorem in measure theory and the Glivenko-Cantelli and Donsker properties. This connection creates…
Near-Gaussian probability densities are common in many important physical applications. Here we develop an asymptotic expansion methodology for computing entropic functionals for such densities. The expansion proposed is a close relative of…
The matrix-based R\'enyi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This intriguing property makes it widely applied in statistical…
A fundamental problem in analysis of complex systems is getting a reliable estimate of entropy of their probability distributions over the state space. This is difficult because unsampled states can contribute substantially to the entropy,…
The Shapley effects are global sensitivity indices: they quantify the impact of each input variable on the output variable in a model. In this work, we suggest new estimators of these sensitivity indices. When the input distribution is…
In this paper, we study the asymptotic distribution of some U-statistics whose entries are functions of empirical moments computed from non-overlapping consecutive blocks of an underlying weakly dependent process. The length of these blocks…
The definition of weighted entropy allows for easy calculation of the entropy of the mixture of measures. In this paper we investigate the problem of equivalent definition of the general entropy function in weighted form. We show that under…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…
For nonlinear supervised learning models, assessing the importance of predictor variables or their interactions is not straightforward because it can vary in the domain of the variables. Importance can be assessed locally with sensitivity…
We address the generalized uncertainty principle in scenarios of successive measurements. Uncertainties are characterized by means of generalized entropies of both the R\'{e}nyi and Tsallis types. Here, specific features of measurements of…
The entropic moments of the probability density of a quantum system in position and momentum spaces describe not only some fundamental and/or experimentally accessible quantities of the system, but also the entropic uncertainty measures of…
Statistical mechanics is generalized on the basis of an additive information theory for incomplete probability distributions. The incomplete normalization $\sum_{i=1}^wp_i^q=1$ is used to obtain generalized entropy $S=-k\sum_{i=1}^wp_i^q\ln…
We provide a general method to analyze the asymptotic properties of a variety of estimators of continuous time diffusion processes when the data are not only discretely sampled in time but the time separating successive observations may…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…