Related papers: Statistical tests based on Renyi entropy estimatio…
The main purpose of this paper is to present new families of test statistics for studying the problem of goodness-of-fit of some data to a latent class model for binary data. The families of test statistics introduced are based on…
In the present follow-up article of a previous one [1] we illustrate the use of the Unconventional Statistical Mechanics described and discussed in the latter. This is done via the analysis, resorting to Renyi approach, of experimental…
Two-sample testing is a fundamental problem in statistics. Despite its long history, there has been renewed interest in this problem with the advent of high-dimensional and complex data. Specifically, in the machine learning literature,…
Renyi's "thinning" operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in…
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine…
A common statistical situation concerns inferring an unknown distribution Q(x) from a known distribution P(y), where X (dimension n), and Y (dimension m) have a known functional relationship. Most commonly, n<m, and the task is relatively…
The paper presents a new statistical method that enables the use of systematic errors in the maximum-likelihood regression of integer-count Poisson data to a parametric model. The method is primarily aimed at the characterization of the…
Many flexible families of positive random variables exhibit non-closed forms of the density and distribution functions and this feature is considered unappealing for modelling purposes. However, such families are often characterized by a…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
This paper proposes several tests of restricted specification in nonparametric instrumental regression. Based on series estimators, test statistics are established that allow for tests of the general model against a parametric or…
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…
We describe five types of results concerning information and concentration of discrete random variables, and relationships between them, motivated by their counterparts in the continuous case. The results we consider are information…
It is possible to derive the maximum entropy principle from thermodynamic stability requirements. Using as a starting point the equilibrium probability distribution, currently used in non-extensive thermostatistics, it turns out that the…
We consider the problem of approximating the empirical Shannon entropy of a high-frequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is…
We investigate the large-time asymptotics of nonlinear diffusion equations $u_t = \Delta u^p$ in dimension $n \ge 1$, in the exponent interval $p > n/(n+2)$, when the initial datum $u_0$ is of bounded second moment. Precise rates of…
Liquid state entropy formulas based on configurational probability distributions are examined for Lennard-Jones fluids across a range temperatures and densities. These formulas are based on expansions of the entropy in series of $n$-body…
We develop an asymptotic theory for $L^2$ norms of sample mean vectors of high-dimensional data. An invariance principle for the $L^2$ norms is derived under conditions that involve a delicate interplay between the dimension $p$, the sample…
The R\'enyi information measures are characterized in terms of their Shannon counterparts, and properties of the former are recovered from first principle via the associated properties of the latter. Motivated by this characterization, a…
We consider two types of entropy, namely, Shannon and R\'{e}nyi entropies of the Poisson distribution, and establish their properties as the functions of intensity parameter. More precisely, we prove that both entropies increase with…
In this paper we derive explicit formulas of the R\'enyi information, Shannon entropy and Song measure for the invariant density of one dimensional ergodic diffusion processes. In particular, the diffusion models considered include the…