Related papers: Strengthening the Entropy Power Inequality
We prove that the reciprocal of Fisher information of a log-concave probability density $X$ in ${\bf{R}}^n$ is concave in $t$ with respect to the addition of a Gaussian noise $Z_t = N(0, tI_n)$. As a byproduct of this result we show that…
We address the problem of applying the Kolmogorov-Sinai method of entropic analysis, expressed in a generalized non-extensive form, to the dynamics of the logistic map at the chaotic threshold, which is known to be characterized by a power…
We study inference with a small labeled sample, a large unlabeled sample, and high-quality predictions from an external model. We link prediction-powered inference with empirical likelihood by stacking supervised estimating equations based…
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…
Several studies demonstrate that there are critical differences between real wireless networks and simulation models. This finding has permitted to extract spatial and temporal properties for links and to provide efficient methods as biased…
In a recent paper `The equi-energy sampler with applications statistical inference and statistical mechanics' [Ann. Stat. 34 (2006) 1581--1619], Kou, Zhou & Wong have presented a new stochastic simulation method called the equi-energy (EE)…
We investigate the optimal performance of dense sensor networks by studying the joint source-channel coding problem. The overall goal of the sensor network is to take measurements from an underlying random process, code and transmit those…
This paper explores some applications of a two-moment inequality for the integral of the $r$-th power of a function, where $0 < r< 1$. The first contribution is an upper bound on the R\'{e}nyi entropy of a random vector in terms of the two…
Weakly almost i.i.d. quantum sources are sequences of multipartite states whose fixed-size marginals converge, on average, to tensor powers of a reference state, while allowing arbitrary global correlations and entanglement. We establish…
Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck…
The data-processing inequality, that is, $I(U;Y) \le I(U;X)$ for a Markov chain $U \to X \to Y$, has been the method of choice for proving impossibility (converse) results in information theory and many other disciplines. Various…
We have calculated the Tsallis entropy and Fisher information matrix (entropy) of spatially-correlated nonextensive systems, by using an analytic non-Gaussian distribution obtained by the maximum entropy method. Effects of the correlated…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
Previous compact representations of permutations have focused on adding a small index on top of the plain data $<\pi(1), \pi(2),...\pi(n)>$, in order to efficiently support the application of the inverse or the iterated permutation. In this…
Following a growing number of studies that, over the past 15 years, have established entropy inequalities via ideas and tools from additive combinatorics, in this work we obtain a number of new bounds for the differential entropy of sums,…
We study the class of self-similar probability density functions with finite mean and variance which maximize R\'{e}nyi's entropy. The investigation is restricted in the Schwartz space $S(\mathbb{R}^d)$ and in the space of…
In ordinary statistical mechanics the Boltzmann-Shannon entropy is related to the Maxwell-Bolzmann distribution $p_i$ by means of a twofold link. The first link is differential and is offered by the Jaynes Maximum Entropy Principle. The…
In this paper, we analyze the indirect source coding problem with side information at both the encoder and decoder, as well as only at the decoder. We first derive structural properties of the two rate distortion functions (RDFs) for…
We produce a series of results extending information-theoretical inequalities (discussed by Dembo--Cover--Thomas in 1989-1991) to a weighted version of entropy. The resulting inequalities involve the Gaussian weighted entropy; they imply a…
Markov Random Field models are powerful tools for the study of complex systems. However, little is known about how the interactions between the elements of such systems are encoded, especially from an information-theoretic perspective. In…