Related papers: Statistical tests based on Renyi entropy estimatio…
Statistical mechanics relies on the complete though probabilistic description of a system in terms of all the microscopic variables. Its object is to derive therefrom static and dynamic properties involving some reduced set of variables.…
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…
Characteristic-function based goodness-of-fit tests are suggested for multivariate observations. The test statistics, which are straightforward to compute, are defined as two-sample criteria measuring discrepancy between multivariate ranks…
Two-sample tests utilizing a similarity graph on observations are useful for high-dimensional and non-Euclidean data due to their flexibility and good performance under a wide range of alternatives. Existing works mainly focused on sparse…
We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given…
We develop the method of Maximum Entropy (ME) as a technique to generate approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
The best-known and most commonly used distribution-property estimation technique uses a plug-in estimator, with empirical frequency replacing the underlying distribution. We present novel linear-time-computable estimators that significantly…
Scientific modeling applications often require estimating a distribution of parameters consistent with a dataset of observations - an inference task also known as source distribution estimation. This problem can be ill-posed, however, since…
We implement a Wang-Landau sampling technique in quantum Monte Carlo (QMC) for the purpose of calculating the Renyi entanglement entropies and associated mutual information. The algorithm converges an estimate for an analogue to the density…
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
We use a Stein identity to define a new class of parametric distributions which we call ``independent additive weighted bias distributions.'' We investigate related $L^2$-type discrepancy measures, empirical versions of which not only…
Spatially referenced data often have autocovariance functions with elliptical isolevel contours, a property known as geometric anisotropy. The anisotropy parameters include the tilt of the ellipse (orientation angle) with respect to a…
Motivated by the orthogonal series density estimation in $L^2([0,1],\mu)$, in this project we consider a new class of functions that we call the approximate sparsity class. This new class is characterized by the rate of decay of the…
In this paper we derive the maximum entropy characteristics of a particular rank order distribution, namely the discrete generalized beta distribution, which has recently been observed to be extremely useful in modelling many several…
The principle of maximum entropy provides a useful method for inferring statistical mechanics models from observations in correlated systems, and is widely used in a variety of fields where accurate data are available. While the assumptions…
In this paper we revisit the Bialynicki-Birula & Mycielski uncertainty principle and its cases of equality. This Shannon entropic version of the well-known Heisenberg uncertainty principle can be used when dealing with variables that admit…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
Bayesian optimal experimental design provides a principled framework for selecting experimental settings that maximize obtained information. In this work, we focus on estimating the expected information gain in the setting where the…
The formalism of statistical mechanics can be generalized by starting from more general measures of information than the Shannon entropy and maximizing those subject to suitable constraints. We discuss some of the most important examples of…
Properties of scalar quantization with $r$th power distortion and constrained R\'enyi entropy of order $\alpha\in (0,1)$ are investigated. For an asymptotically (high-rate) optimal sequence of quantizers, the contribution to the R\'enyi…