Related papers: Deformed Statistics Formulation of the Information…
This paper introduces the basic concepts of information theory. Based on these concepts, we regard the states in the state space and the types of ideal gases as the symbols in a symbol set to calculate the mixing entropy of ideal gas…
Relevance-based explanation is a scheme in which partial assignments to Bayesian belief network variables are explanations (abductive conclusions). We allow variables to remain unassigned in explanations as long as they are irrelevant to…
We find the value of constants related to constraints in characterization of some known statistical distributions and then we proceed to use the idea behind maximum entropy principle to derive generalized version of this distributions using…
In this work we propose a statistical approach to handling sources of theoretical uncertainty in string theory models of inflation. By viewing a model of inflation as a probabilistic graph, we show that there is an inevitable information…
The general formalisms of the $q$-dual statistics, the Boltzmann-Gibbs statistics, and three versions of the Tsallis statistics known as Tsallis-1, Tsallis-2, and Tsallis-3 statistics have been considered in the canonical ensemble. We have…
In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that…
This paper applies the recently axiomatized Optimum Information Principle (minimize the Kullback-Leibler information subject to all relevant information) to nonparametric density estimation, which provides a theoretical foundation as well…
The standard formulation of thermostatistics, being based on the Boltzmann-Gibbs distribution and logarithmic Shannon entropy, describes idealized uncorrelated systems with extensive energies and short-range interactions. In this letter, we…
The partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem,…
The optimization problems defining meta-stable or stationary equilibrium are explored. The Gibbs scheme is modified aiming to describe the statistical properties of a class of non-equilibrium and metastable states. The system is assumed to…
We present a general non-parametric statistical inference theory for integrals of quantiles without assuming any specific sampling design or dependence structure. Technical considerations are accompanied by examples and discussions,…
While mutual information effectively quantifies dependence between two variables, it does not by itself reveal the complex, fine-grained interactions among variables, i.e., how multiple sources contribute redundantly, uniquely, or…
We generalize the Shannon's information theory in a nonadditive way by focusing on the source coding theorem. The nonadditive information content we adopted is consistent with the concept of the form invariance structure of the nonextensive…
The feature attribution method reveals the contribution of input variables to the decision-making process to provide an attribution map for explanation. Existing methods grounded on the information bottleneck principle compute information…
Tsallis statistics (or q-statistics) in nonextensive statistical mechanics is a one-parameter description of correlated states. In this paper we use a translated entropic index: $1 - q \to q$ . The essence of this translation is to improve…
The problem of distributed representation learning is one in which multiple sources of information $X_1,\ldots,X_K$ are processed separately so as to learn as much information as possible about some ground truth $Y$. We investigate this…
Through the generalization of Khinchin's classical axiomatic foundation, a basis is developed for nonadditive information theory. The classical nonadditive conditional entropy indexed by the positive parameter q is introduced and then…
Double-descent refers to the unexpected drop in test loss of a learning algorithm beyond an interpolating threshold with over-parameterization, which is not predicted by information criteria in their classical forms due to the limitations…
We formulate a new information-theoretic principle--the shifted composition rule--which bounds the divergence (e.g., Kullback-Leibler or R\'enyi) between the laws of two stochastic processes via the introduction of auxiliary shifts. In this…
The nonextensive statistics based on Tsallis entropy have been so far used for the systems composed of subsystems having same $q$. The applicability of this statistics to the systems with different $q$'s is still a matter of investigation.…