Related papers: R\'{e}nyi Cross-Entropy Measures for Common Distri…
It is well-known that the Shannon entropies of some parameterized probability distributions are concave functions with respect to the parameter. In this paper we consider a family of such distributions (including the binomial, Poisson, and…
We consider the two-dimensional (2d) Ising model on a infinitely long cylinder and study the probabilities $p_i$ to observe a given spin configuration $i$ along a circular section of the cylinder. These probabilities also occur as…
The random variable simulation problem consists in using a $k$-dimensional i.i.d. random vector $X^{k}$ with distribution $P_{X}^{k}$ to simulate an $n$-dimensional i.i.d. random vector $Y^{n}$ so that its distribution is approximately…
When incrementally trained on new classes, deep neural networks are subject to catastrophic forgetting which leads to an extreme deterioration of their performance on the old classes while learning the new ones. Using a small memory…
We investigate the efficiency of several types of continued fraction expansions of a number in the unit interval using a generalization of Lochs theorem from 1964. Thus, we aimed to compare the efficiency by describing the rate at which the…
In this paper we introduce a biparametric family of transformations which can be seen as an extension of the so-called up and down transformations. This new class of transformations allows to us to introduce new informational functionals,…
In this paper, we focus on the separability of classes with the cross-entropy loss function for classification problems by theoretically analyzing the intra-class distance and inter-class distance (i.e. the distance between any two points…
The Tsallis and R\'enyi entropies are important quantities in the information theory, statistics and related fields because the Tsallis entropy is an one parameter generalization of the Shannon entropy and the R\'enyi entropy includes…
The ultimate purpose of the statistical analysis of ordinal patterns is to characterize the distribution of the features they induce. In particular, knowing the joint distribution of the pair Entropy-Statistical Complexity for a large class…
We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent. The minimum entropy required for such a latent is known as common entropy in information theory. We…
Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…
In the online prediction framework, we use generalized entropy of to study the loss rate of predictors when outcomes are drawn according to stationary ergodic distributions over the binary alphabet. We show that the notion of generalized…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
Lower bounds for the R\'enyi entropies of sums of independent random variables taking values in cyclic groups of prime order under permutations are established. The main ingredients of our approach are extended rearrangement inequalities in…
We introduce a pair of transformations, which are mutually inverse, acting on rather general classes of probability densities in R. These transformations have the property of interchanging the main informational measures such as p-moments,…
We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed…
The study of quantum and classical correlations between subsystems is fundamental to understanding many-body physics. In quantum information theory, the quantum mutual information, $I(A;B)$, is a measure of correlation between the…
In this work, we study the real-time evolution of pseudo-(R\'enyi) entropy, a generalization of entanglement entropy, in two-dimensional conformal field theories (CFTs). We focus on states obtained by acting primary operators located at…
We derive a novel chain rule for a family of channel conditional entropies, covering von Neumann and sandwiched R\'{e}nyi entropies. In the process, we show that these channel conditional entropies are equal to their regularized version,…
Recent end-to-end deep neural networks for disparity regression have achieved the state-of-the-art performance. However, many well-acknowledged specific properties of disparity estimation are omitted in these deep learning algorithms.…