Related papers: Generalized informational functionals and new mono…
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual…
In this paper, we provide the R\'enyi entropy and complexity measure for a novel, flexible class of skew-gaussian distributions and their related families, as a characteristic form of the skew-gaussian Shannon entropy. We give closed…
We investigate the problem of representing information measures in terms of the moments of the underlying random variables. First, we derive polynomial approximations of the conditional expectation operator. We then apply these…
The paper investigates the problem of performing correlation analysis when the number of observations is very large. In such a case, it is often necessary to combine the random observations to achieve dimensionality reduction of the…
Researchers often misinterpret and misrepresent statistical outputs. This abuse has led to a large literature on modification or replacement of testing thresholds and $P$-values with confidence intervals, Bayes factors, and other devices.…
Using the generalized entropies which depend on two parameters we propose a set of quantitative characteristics derived from the Information Geometry based on these entropies. Our aim, at this stage, is modest, as we are first constructing…
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We…
We propose a dimension reduction framework for feature extraction and moment reconstruction in dynamical systems that operates on spaces of probability measures induced by observables of the system rather than directly in the original data…
We study the multifractal analysis of self-similar measures arising from random homogeneous iterated function systems. Under the assumption of the uniform strong separation condition, we see that this analysis parallels that of the…
Shannon entropy and Fisher information functionals are known to quantify certain information-theoretic properties of continuous probability distributions of various origins. We carry out a systematic study of these functionals, while…
We introduce a new sufficient dimension reduction framework that targets a statistical functional of interest, and propose an efficient estimator for the semiparametric estimation problems of this type. The statistical functional covers a…
In this paper we derive explicit formulas of the R\'enyi information, Shannon entropy and Song measure for the invariant density of one dimensional ergodic diffusion processes. In particular, the diffusion models considered include the…
Competing styles of Statistical Mechanics have been introduced as practical succedaneous to the conventional well established Boltzmann-Gibbs statistical mechanics, when in the use of the latter the researcher is impaired in his/her…
We investigate certain optimization problems for Shannon information measures, namely, minimization of joint and conditional entropies $H(X,Y)$, $H(X|Y)$, $H(Y|X)$, and maximization of mutual information $I(X;Y)$, over convex regions. When…
The information in an individual finite object (like a binary string) is commonly measured by its Kolmogorov complexity. One can divide that information into two parts: the information accounting for the useful regularity present in the…
Information theoretic quantities play an important role in various settings in machine learning, including causality testing, structure inference in graphical models, time-series problems, feature selection as well as in providing privacy…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
The main purpose of this article is to provide a common generalization of the notions of a topological and Kolmogorov-Sinai entropy for arbitrary representations of discrete amenable groups on objects of (abstract) categories. This is…
First, we present a concise glossary of formulas for composition of standard, cumulant, factorial, and factorial cumulant moments in superposition (compound) models, where final particles are created via independent emission from a…
This is a paper in the intersection of time series analysis and complexity theory that presents new results on permutation complexity in general and permutation entropy in particular. In this context, permutation complexity refers to the…