Related papers: Information theory and generalized statistics
Researchers have developed ways to generalize the mean and variance to situations in which a data metric is available. We apply the tools developed in Pennec (2006) to categorical data, and show the generality of this approach by…
Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its…
By using the Renyi entropy, and following the same scheme that in the Fisher-Renyi entropy product case, a generalized statistical complexity is defined. Several properties of it, including inequalities and lower and upper bounds are…
This paper proposes a theoretical framework which models the information provided by retrieval systems in terms of Information Theory. The proposed framework allows to formalize: (i) system effectiveness as an information theoretic…
In this paper the author analyses the weighted Renyi entropy in order to derive several inequalities in weighted case. Furthermore, using the proposed notions $\alpha$-th generalized derivation and ($\alpha$; p)-th weighted Fisher…
Practical data analysis involves many implicit or explicit assumptions about the good behavior of the data, and excludes consideration of various potentially pathological or limit cases. In this work, we present a new general theory of…
We have discussed dynamical properties of the Tsallis entropy and the generalized Fisher information in nonextensive systems described by the Langevin model subjected to additive and multiplicative noise. Analytical expressions for the…
We propose a unifying picture where the notion of generalized entropy is related to information theory by means of a group-theoretical approach. The group structure comes from the requirement that an entropy be well defined with respect to…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
Competing styles of Statistical Mechanics have been introduced as practical succedaneous to the conventional well established Boltzmann-Gibbs statistical mechanics, when in the use of the latter the researcher is impaired in his/her…
The generalized Gaussian distribution that stems from information theory is studied. The log-Minkowski problem associated with generalized Gaussian distribution shall be introduced and solved.
Firstly, assuming Gaussianity, equations for the following information theory measures are presented: total correlation/coherence (TC), dual total correlation/coherence (DTC), O-information, TSE complexity, and redundancy-synergy index…
We consider the self-similar phase space with reduced fractal dimension $d$ being distributed within domain $0<d<1$ with spectrum $f(d)$. Related thermostatistics is shown to be governed by the Tsallis' formalism of the non-extensive…
In this article, we discuss the problem of establishing relations between information measures assessed for network structures. Two types of entropy based measures namely, the Shannon entropy and its generalization, the R\'{e}nyi entropy…
We show that extensive thermostatistics based on Renyi entropy and Kolmogorov-Nagumo averages can be expressed in terms of Tsallis non- extensive thermostatistics. We use this correspondence to generalize thermostatistics to a large class…
Information theory is a powerful framework to capture aspects of dynamical systems with multiple degrees of freedom. Mathematically, the dynamics can be represented as a continuous curve $\mathcal{C}$ on a suitable hyperplane in flat space…
In this work, we derive information-theoretic properties for a modified Tsallis entropy, hereinafter referred to as q-entropy. We introduce the notions of joint q-entropy, conditional q-entropy, relative q-entropy, conditional mutual…
The generalized fault diagram, a data structure for failure analysis based on the influence diagram, is defined. Unlike the fault tree, this structure allows for dependence among the basic events and replicated logical elements. A heuristic…