Related papers: Measuring Information from Moments
Recent studies found that many channels are affected by additive noise that is impulsive in nature and is best explained by heavy-tailed symmetric alpha-stable distributions. Dealing with impulsive noise environments comes with an added…
Due to increased awareness of data protection and corresponding laws many data, especially involving sensitive personal information, are not publicly accessible. Accordingly, many data collecting agencies only release aggregated data, e.g.…
The Maximum Mutual Information (MMI) criterion is different from the Least Error Rate (LER) criterion. It can reduce failing to report small probability events. This paper introduces the Channels Matching (CM) algorithm for the MMI…
If moments of singular measures are passed as inputs to the entropy maximization procedure, the optimization algorithm might not terminate. The framework developed in our previous paper demonstrated how input moments of measures, on a broad…
We introduce new classes of informational functionals, called \emph{upper moments}, respectively \emph{down-Fisher measures}, obtained by applying classical functionals such as $p$-moments and the Fisher information to the recently…
Denoising diffusion models have spurred significant gains in density modeling and image generation, precipitating an industrial revolution in text-guided AI art generation. We introduce a new mathematical foundation for diffusion models…
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on…
We define two minimum distance estimators for dependent data by minimizing some approximated Maximum Mean Discrepancy distances between the true empirical distribution of observations and their assumed (parametric) model distribution. When…
This work considers the computation of risk measures for quantities of interest governed by PDEs with Gaussian random field parameters using Taylor approximations. While efficient, Taylor approximations are local to the point of expansion,…
This paper investigates achievable information rates and error exponents of mismatched decoding when the channel belongs to the class of channels that are close to the decoding metric in terms of relative entropy. For both discrete- and…
This paper considers a Gaussian channel with one transmitter and two receivers. The goal is to maximize the communication rate at the intended/primary receiver subject to a disturbance constraint at the unintended/secondary receiver. The…
We endeavour to estimate numerous multi-dimensional means of various probability distributions on a common space based on independent samples. Our approach involves forming estimators through convex combinations of empirical means derived…
We consider a problem of statistical mean estimation in which the samples are not observed directly, but are instead observed by a relay (``teacher'') that transmits information through a memoryless channel to the decoder (``student''), who…
We consider a channel $Y=X+N$ where $X$ is a random variable satisfying $\mathbb{E}[|X|]<\infty$ and $N$ is an independent standard normal random variable. We show that the minimum mean-square error estimator of $X$ from $Y,$ which is given…
We develop new unbiased estimators of a number of quantities defined for functions of conditional moments, like conditional expectations and variances, of functions of two independent random variables given the first variable, including…
The persistent mutual information (PMI) is a complexity measure for stochastic processes. It is related to well-known complexity measures like excess entropy or statistical complexity. Essentially it is a variation of the excess entropy so…
The more information a measurement provides about a quantum system's position statistics, the less information a subsequent measurement can provide about the system's momentum statistics. This information trade-off is embodied in the…
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…
We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying distribution. This quantity is based on a recently proposed…
In this paper, we propose a new measure for the freshness of information, which uses the mutual information between the real-time source value and the delivered samples at the receiver to quantify the freshness of the information contained…