Related papers: The Relative Information Generating Function-A Qua…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
We address quantum estimation in situations where one has at disposal data from the measurement of an incomplete set of observables and some a priori information on the state itself. By expressing the a priori information in terms of a bias…
In this paper we derive an integral (with respect to time) representation of the relative entropy (or Kullback-Leibler Divergence) between measures mu and P on the space of continuous functions from time 0 to T. The underlying measure P is…
Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably…
The object of this paper is to introduce a new weighted information generating function whose derivative at point 1 gives some well known measures of information. Some properties and particular cases of the proposed generating function have…
This paper is concerned with non-parametric estimation of the entropy in ranked set sampling. Theoretical properties of the proposed estimator are studied. The proposed estimator is compared with the rival estimator in simple random…
Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty quantification. In this paper, we develop a method for comparing how close different generative…
The past entropy is considered as an uncertainty measure for the past lifetime distribution. Generating function approach to entropy become popular in recent time as it generate several well-known entropy measures. In this paper, we…
The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understand by examining the equation, an intuition and understanding…
This work presents an upper-bound to value that the Kullback-Leibler (KL) divergence can reach for a class of probability distributions called quantum distributions (QD). The aim is to find a distribution $U$ which maximizes the KL…
A generalized Kullback-Leibler relative entropy is introduced starting with the symmetric Jackson derivative of the generalized overlap between two probability distributions. The generalization retains much of the structure possessed by the…
We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…
We review the properties of the quantum relative entropy function and discuss its application to problems of classical and quantum information transfer and to quantum data compression. We then outline further uses of relative entropy to…
We show how convergence to the Gumbel distribution in an extreme value setting can be understood in an information-theoretic sense. We introduce a new type of score function which behaves well under the maximum operation, and which implies…
Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle…
This paper applies the recently axiomatized Optimum Information Principle (minimize the Kullback-Leibler information subject to all relevant information) to nonparametric density estimation, which provides a theoretical foundation as well…
Quantum relative entropy, a quantum generalization of the renowned Kullback-Leibler divergence, serves as a fundamental measure of the distinguishability between quantum states and plays a pivotal role in quantum information science.…
A well-known technique in estimating probabilities of rare events in general and in information theory in particular (used, e.g., in the sphere-packing bound), is that of finding a reference probability measure under which the event of…
In a variety of applications it is important to extract information from a probability measure $\mu$ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and possibly conditioned) continuous time…
Here, we investigate the uncertainty of dynamical observables in classical systems manipulated by repeated measurements and feedback control; the precision should be enhanced in the presence of an external controller but limited by the…