Related papers: Information functionals and the notion of (un)cert…
We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…
We define the concept of dependence among multiple variables using maximum entropy techniques and introduce a graphical notation to denote the dependencies. Direct inference of information theoretic quantities from data uncovers…
Uncertainty relations based on information theory for both discrete and continuous distribution functions are briefly reviewed. We extend these results to account for (differential) R\'{e}nyi entropy and its related entropy power. This…
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
Let f_1,f_2,..., be functions chosen independently and uniformly from the set of all functions from a set of cardinality n into itself. Let g_t be the composition of the first t functions, and let T be the smallest t for which g_t is…
The information causality principle is a generalisation of the no-signalling principle which implies some of the known restrictions on quantum correlations. But despite its clear physical motivation, information causality is formulated in…
We study the informational underpinnings of thermodynamics and statistical mechanics, using an abstract framework, general probabilistic theories, capable of describing arbitrary physical theories. This allows one to abstract the…
This paper revisits Brownian motion from the perspective of Information Theory, aiming to explore the connections between Information Theory, Thermodynamics, and Complex Science. First, we propose a single-particle discrete Brownian motion…
The neutron distribution of neutron-rich nuclei provides critical information on the structure of finite nuclei and neutron stars. Parity violating experiments -- such as PREX and CREX -- provide a clean and largely model-independent…
Parametric inference posits a statistical model that is a specified family of probability distributions. Restricted inference, e.g., restricted likelihood ratio testing, attempts to exploit the structure of a statistical submodel that is a…
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
Herein we define a measure of similarity between classification distributions that is both principled from the perspective of statistical pattern recognition and useful from the perspective of machine learning practitioners. In particular,…
The most fundamental problem in statistics is the inference of an unknown probability distribution from a finite number of samples. For a specific observed data set, answers to the following questions would be desirable: (1) Estimation:…
Bayesian inference is often utilized for uncertainty quantification tasks. A recent analysis by Xu and Raginsky 2022 rigorously decomposed the predictive uncertainty in Bayesian inference into two uncertainties, called aleatoric and…
The fundamental properties of quantum information and its applications to computing and cryptography have been greatly illuminated by considering information-theoretic tasks that are provably possible or impossible within non-relativistic…
Quadrature rules estimate the value of an integral when the function is given by a table of values. Every binary string defines a quadrature rule by choosing which endpoint of each interval represents the interval. The standard rules, such…
This study intends to introduce kernel mean embedding of probability measures over infinite-dimensional separable Hilbert spaces induced by functional response statistical models. The embedded function represents the concentration of…
This paper introduces a framework for modeling cyclical and feedback-driven information flow through a generalized family of entropy-modulated transformations called derangetropy functionals. Unlike scalar and static entropy measures such…
We consider probabilistic theories in which the most elementary system, a two-dimensional system, contains one bit of information. The bit is assumed to be contained in any complete set of mutually complementary measurements. The…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…