Related papers: On the effectiveness of a binless entropy estimato…
Estimators of information theoretic measures such as entropy and mutual information are a basic workhorse for many downstream applications in modern data science. State of the art approaches have been either geometric (nearest neighbor (NN)…
It is by now well known that the Boltzmann-Gibbs (BG) entropy $S_{BG}=-k\sum_{i=1}^W p_i \ln p_i$ can be usefully generalized into the entropy $S_q=k (1-\sum_{i=1}^Wp_i^{q}) / (q-1)$ ($q\in \mathcal{R}; S_1=S_{BG}$). Microscopic dynamics…
We analyze the extreme value dependence of independent, not necessarily identically distributed multivariate regularly varying random vectors. More specifically, we propose estimators of the spectral measure locally at some time point and…
In this paper we extend Fischler's quantitative generalization of Nesterenko's linear independence criterion, by weakening the hypotheses on the divisors of the coe cients of the linear forms and allowing (to some extent) the linear forms…
We consider a general multivariate model where univariate marginal distributions are known up to a parameter vector and we are interested in estimating that parameter vector without specifying the joint distribution, except for the…
We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, R\'enyi, generalized R\'enyi, Tsallis, Sharma-Mittal, and modified Shannon entropy,…
We establish several Schur-convexity type results under fixed variance for weighted sums of independent gamma random variables and obtain nonasymptotic bounds on their R\'enyi entropies. In particular, this pertains to the recent results by…
A method for estimating the Shannon differential entropy of multidimensional random variables using independent samples is described. The method is based on decomposing the distribution into a product of the marginal distributions and the…
The title of the paper leads to an incorrect conclusion as we show that the equilibrium result of the paper is a special limit of a general result for nonequilibrium systems in internal equilibrium already available in the literature. We…
The entropic lattice Boltzmann framework proposed the construction of the discrete equilibrium by taking into consideration minimization of a discrete entropy functional. The effect of this form of the discrete equilibrium on properties of…
Tsallis' non-extensive entropy is extended to incorporate the dependence on affinities between the microstates of a system. At the core of our construction of the extended entropy ($\mathcal{H}$) is the concept of the effective number of…
We study the Lp-integrated risk of some classical estimators of the density, when the observations are drawn from a strictly stationary sequence. The results apply to a large class of sequences, which can be non-mixing in the sense of…
The paper studies the problem of distributed parameter estimation in multi-agent networks with exponential family observation statistics. A certainty-equivalence type distributed estimator of the consensus + innovations form is proposed in…
We show that the generalized Boltzmann distribution is the only distribution for which the Gibbs-Shannon entropy equals the thermodynamic entropy. This result means that the thermodynamic entropy and the Gibbs-Shannon entropy are not…
Increasing the number $N$ of elements of a system typically makes the entropy to increase. The question arises on {\it what particular entropic form} we have in mind and {\it how it increases} with $N$. Thermodynamically speaking it makes…
Point estimators may not exist, need not be unique, and their distributions are not parameter invariant. Generalized estimators provide distributions that are parameter invariant, unique, and exist when point estimates do not. Comparing…
We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…
This study shows that the generalized Boltzmann distribution is the only distribution mathematically consistent with thermodynamics when the system is described by an ensemble of a certain mathematical form. This mathematical form is very…
We consider a nonparametric regression model with continuous endogenous independent variables when only discrete instruments are available that are independent of the error term. Although this framework is very relevant for applied…
We provide finite-sample distribution approximations, that are uniform in the parameter, for inference in linear mixed models. Focus is on variances and covariances of random effects in cases where existing theory fails because their…