Related papers: Some Generalized Information and Divergence Genera…
The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the…
Entanglement criteria for an $n$-partite quantum system with continuous variables are formulated in terms of R\'{e}nyi entropies. R\'{e}nyi entropies are widely used as a good information measure due to many nice properties. Derived…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
This paper proposes a new method for estimating the joint probability mass function of a pair of discrete random variables. This estimator is used to construct joint Shannon R\'enyi-Tsallis entropies, and the mutual information estimates of…
Estimating statistical properties is fundamental in statistics and computer science. In this paper, we propose a unified quantum algorithm framework for estimating properties of discrete probability distributions, with estimating R\'enyi…
Information functionals allow to quantify the degree of randomness of a given probability distribution, either absolutely (through min/max entropy principles) or relative to a prescribed reference one. Our primary aim is to analyze the…
Starting from the geometrical interpretation of the R\'enyi entropy, we introduce further extensive generalizations and study their properties. In particular, we found the probability distribution function obtained by the MaxEnt principle…
Information theoretic quantities play an important role in various settings in machine learning, including causality testing, structure inference in graphical models, time-series problems, feature selection as well as in providing privacy…
Channel simulation is to simulate a noisy channel using noiseless channels with unlimited shared randomness. This can be interpreted as the reverse problem to Shannon's noisy coding theorem. In contrast to previous works, our approach…
Recursive Bayesian inference (RBI) provides optimal Bayesian latent variable estimates in real-time settings with streaming noisy observations. Active RBI attempts to effectively select queries that lead to more informative observations to…
We consider the two-dimensional (2d) Ising model on a infinitely long cylinder and study the probabilities $p_i$ to observe a given spin configuration $i$ along a circular section of the cylinder. These probabilities also occur as…
We generalize the Point information gain (PIG) and derived quantities, i.e. Point information entropy (PIE) and Point information entropy density (PIED), for the case of R\'enyi entropy and simulate the behavior of PIG for typical…
In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
We study the $p$-R\'{e}nyi entropy power inequality with a weight factor $t$ on two independent continuous random variables $X$ and $Y$. The extension essentially relies on a modulation on the sharp Young's inequality due to Bobkov and…
In R\'enyi's representation for exponential order statistics, we replace the iid exponential sequence with any iid sequence, and call the resulting order statistic generalized R\'enyi statistic. We prove that by randomly reordering the…
We associate to the p-th R\'enyi entropy a definition of entropy power, which is the natural extension of Shannon's entropy power and exhibits a nice behaviour along solutions to the p-nonlinear heat equation in $R^n$. We show that the…
Information measures can be constructed from R\'enyi divergences much like mutual information from Kullback-Leibler divergence. One such information measure is known as Sibson $\alpha$-mutual information and has received renewed attention…
The entropy accumulation theorem, and its subsequent generalized version, is a powerful tool in the security analysis of many device-dependent and device-independent cryptography protocols. However, it has the drawback that the finite-size…