Related papers: Error Bounds on a Mixed Entropy Inequality
Following [1], the aim of this paper is to analyze the relative weighted entropy involving the central moments weight functions. We compare the standard relative entropy with the weighted case in two particular forms of Gaussian…
Gaussian and Laplacian entropy models are proved effective in learned point cloud attribute compression, as they assist in arithmetic coding of latents. However, we demonstrate through experiments that there is still unutilized information…
Shearer's inequality bounds the sum of joint entropies of random variables in terms of the total joint entropy. We give another lower bound for the same sum in terms of the individual entropies when the variables are functions of…
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensional Euclidean space,…
The time decay of fully discrete finite-volume approximations of porous-medium and fast-diffusion equations with Neumann or periodic boundary conditions is proved in the entropy sense. The algebraic or exponential decay rates are computed…
Sub-Gaussian and subexponential distributions are introduced and applied to study the fluctuation-response relation out of equilibrium. A bound on the difference in expected values of an arbitrary sub-Gaussian or subexponential physical…
It is observed that the entropy reduction (the information gain in the initial terminology) of an efficient (ideal or pure) quantum measurement coincides with the generalized quantum mutual information of a q-c channel mapping an a priori…
We uncover the quantum fluctuation-response inequality, which, in the most general setting, establishes a bound for the mean difference of an observable at two different quantum states, in terms of the quantum relative entropy. When the…
The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have…
Concentration inequalities for the sample mean, like those due to Bernstein, Hoeffding, and Bentkus, are valid for any sample size but overly conservative, yielding confidence intervals that are unnecessarily wide. The central limit theorem…
This document expands upon the relationship between discrete and continuous entropy given in (Phys. Rev. Lett. 110 130407), \Violating Continuous Variable Einstein-Podolsky-Rosen Steering with Discrete Measurements". We provide a detailed…
We study the problem of identifying the causal relationship between two discrete random variables from observational data. We recently proposed a novel framework called entropic causality that works in a very general functional model but…
Bayesian optimal experimental design provides a principled framework for selecting experimental settings that maximize obtained information. In this work, we focus on estimating the expected information gain in the setting where the…
When factorized approximations are used for variational inference (VI), they tend to underestimate the uncertainty -- as measured in various ways -- of the distributions they are meant to approximate. We consider two popular ways to measure…
This paper first focuses on deriving an alternative approach for proving an extremal entropy inequality (EEI), originally presented in [11]. The proposed approach does not rely on the channel enhancement technique, and has the advantage…
In this paper, both non-mixing and mixing local minima of the entropy are analyzed from the viewpoint of blind source separation (BSS); they correspond respectively to acceptable and spurious solutions of the BSS problem. The contribution…
We produce a series of results extending information-theoretical inequalities (discussed by Dembo--Cover--Thomas in 1989-1991) to a weighted version of entropy. The resulting inequalities involve the Gaussian weighted entropy; they imply a…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
This paper is devoted to the study of the properties of entropy as a function of the Hurst index, which corresponds to the fractional Gaussian noise. Since the entropy of the Gaussian vector depends on the determinant of the covariance…