Related papers: A Probabilistic Upper Bound on Differential Entrop…

We initiate a systematic investigation of distribution testing in the framework of algorithmic replicability. Specifically, given independent samples from a collection of probability distributions, the goal is to characterize the sample…

We define the entropic bounds, i.e minimal uncertainty for pairs of unitary testers in distinguishing between unitary transformations not unlike the well known entropic bounds for observables. We show that in the case of specific sets of…

This brief paper develops a probability density that models processes for which the physical mechanism is unknown. It has desirable properties which are not realized by densities derived from Gaussian process or other classic methods. In…

Entropy is a central concept in physics, but can be challenging to calculate even for systems that are easily simulated. This is exacerbated out of equilibrium, where generally little is known about the distribution characterizing simulated…

Given a sample of independent and identically distributed random variables, a novel nonparametric maximum entropy method is presented to estimate the underlying continuous univariate probability density function (pdf). Estimates are found…

As a part of the construction of an information theory based on general probabilistic theories, we propose and investigate the several distinguishability measures and "entropies" in general probabilistic theories. As their applications,…

We study the problem of testing discrete distributions with a focus on the high probability regime. Specifically, given samples from one or more discrete distributions, a property $\mathcal{P}$, and parameters $0< \epsilon, \delta <1$, we…

An index of uniformity is developed as an alternative to the maximum-entropy principle for selecting continuous, differentiable probability distributions $\mathcal{P}$ subject to constraints $C$. The uniformity index developed in this paper…

Tight lower and upper bounds on the ratio of relative entropies of two probability distributions with respect to a common third one are established, where the three distributions are collinear in the standard $(n-1)$-simplex. These bounds…

Characterising the capacity region for a network can be extremely difficult, especially when the sources are dependent. Most existing computable outer bounds are relaxations of the Linear Programming bound. One main challenge to extend…

Organisms and algorithms learn probability distributions from previous observations, either over evolutionary time or on the fly. In the absence of regularities, estimating the underlying distribution from data would require observing each…

Anomalies are strange data points; they usually represent an unusual occurrence. Anomaly detection is presented from the perspective of Wireless sensor networks. Different approaches have been taken in the past, as we will see, not only to…

This paper considers the entropy of the sum of (possibly dependent and non-identically distributed) Bernoulli random variables. Upper bounds on the error that follows from an approximation of this entropy by the entropy of a Poisson random…

In this letter, we give a concise, closed-form expression for the differential entropy of the sum of two independent, non-identically-distributed exponential random variables. The derivation is straightforward, but such a concise entropy…

In this paper, we address the probabilistic error quantification of a general class of prediction methods. We consider a given prediction model and show how to obtain, through a sample-based approach, a probabilistic upper bound on the…

Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…

Probability distributions and densities are derived for the excess and deficiency of the intensity or instantaneous energy (quasi-static power) associated with a $p$-dimensional random vector field. Explicit expressions for the exact…

In this work, we study the generalization capability of algorithms from an information-theoretic perspective. It has been shown that the expected generalization error of an algorithm is bounded from above by a function of the relative…

We propose a new way of defining entropy of a system, which gives a general form which may be nonextensive as Tsallis entropy, but is linearly dependent on component entropies, like Renyi entropy, which is extensive. This entropy has a…

The uncertainty principle sets a bound on our ability to predict the measurement outcomes of two incompatible observables which are measured on a quantum particle simultaneously. In quantum information theory, the uncertainty principle can…