Related papers: Generalized informational functionals and new mono…
We evaluate the asymptotics of equivocations, their exponents as well as their second-order coding rates under various R\'{e}nyi information measures. Specifically, we consider the effect of applying a hash function on a source and we…
We derive Bose-Einstein statistics and Fermi-Dirac statistics by Principle of Maximum Entropy applied to two families of entropy functions different from the Boltzmann-Gibbs-Shannon entropy. These entropy functions are identified with…
In this paper, we propose a new class of distributions by exponentiating the random variables associated with the probability density functions of composite distributions. We also derive some mathematical properties of this new class of…
In the past over two decades, very fruitful results have been obtained in information theory in the study of the Shannon entropy. This study has led to the discovery of a new class of constraints on the Shannon entropy called…
We present a categorical viewpoint of probability measures by showing that a probability measure can be viewed as a weakly averaging affine measurable functional taking values in the unit interval which preserves limits. The probability…
Existing polarization theories have mostly been concerned with Shannon's information measures, such as Shannon entropy and mutual information, and some related measures such as the Bhattacharyya parameter. In this work, we extend…
On account of a greater need for understanding the complexity of time series like physiological time series, financial time series, and many more that enter into picture for their inculpation with real-world problems, several complexity…
Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond…
Shannon entropy is a polymatroidal set function and lies at the foundation of information theory, yet the class of entropic polymatroids is strictly smaller than the class of all submodular functions. In parallel, submodular and…
This is an elementary introduction to infinite-dimensional probability. In the lectures, we compute the exact mean values of some functionals on C[0,1] and L[0,1] by considering these functionals as infinite-dimensional random variables.…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
Entropy and its various generalizations are important in many fields, including mathematical statistics, communication theory, physics and computer science, for characterizing the amount of information associated with a probability…
We classify all functions which, when applied term by term, leave invariant the sequences of moments of positive measures on the real line. Rather unexpectedly, these functions are built of absolutely monotonic components, or reflections of…
This article expands the framework of Bayesian inference and provides direct probabilistic methods for approaching inference tasks that are typically handled with information theory. We treat Bayesian probability updating as a random…
We introduce and discuss the notion of monotonicity for the complexity measures of general probability distributions, patterned after the resource theory of quantum entanglement. Then, we explore whether this property is satisfied by the…
The basic properties of the Fisher information allow to reveal the statistical meaning of classical inequalities between mean functions. The properties applied to scale mixtures of Gaussian distributions lead to a new mean function of…
Concepts used in the scientific study of complex systems have become so widespread that their use and abuse has led to ambiguity and confusion in their meaning. In this paper we use information theory to provide abstract and concise…
In this work, we propose two information generating functions: general weighted information and relative information generating functions, and study their properties. { It is shown that the general weighted information generating function…
Predictive inference requires balancing statistical accuracy against informational complexity, yet the choice of complexity measure is usually imposed rather than derived. We treat econometric objects as predictive rules, mappings from…
We address the problem of quantifying the information content of a source for an arbitrary information theory, where the information content is defined in terms of the asymptotic achievable compression rate. The functions that solve this…