Related papers: Point Information Gain and Multidimensional Data A…
Deep learning has become the standard approach for most machine learning tasks. While its impact is undeniable, interpreting the predictions of deep learning models from a human perspective remains a challenge. In contrast to model…
Starting from the geometrical interpretation of the R\'enyi entropy, we introduce further extensive generalizations and study their properties. In particular, we found the probability distribution function obtained by the MaxEnt principle…
The Partial Information Decomposition (PID) framework has emerged as a powerful tool for analyzing high-order interdependencies in complex network systems. However, its application to dynamic processes remains challenging due to the…
While point cloud semantic segmentation is a significant task in 3D scene understanding, this task demands a time-consuming process of fully annotating labels. To address this problem, recent studies adopt a weakly supervised learning…
Information-theoretic quantities, such as entropy, are used to quantify the amount of information a given variable provides. Entropies can be used together to compute the mutual information, which quantifies the amount of information two…
The entropy of a pair of random variables is commonly depicted using a Venn diagram. This representation is potentially misleading, however, since the multivariate mutual information can be negative. This paper presents new measures of…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
A central issue of the science of complex systems is the quantitative characterization of complexity. In the present work we address this issue by resorting to information geometry. Actually we propose a constructive way to associate to a -…
Topological features such as persistence diagrams and their functional approximations like persistence images (PIs) have been showing substantial promise for machine learning and computer vision applications. This is greatly attributed to…
We discuss here the use of generalized forms of entropy, taken as information measures, to characterize phase transitions and critical behavior in thermodynamic systems. Our study is based on geometric considerations pertaining to the space…
Information theoretic quantities play an important role in various settings in machine learning, including causality testing, structure inference in graphical models, time-series problems, feature selection as well as in providing privacy…
According to E.T. Jaynes and E.P. Wigner, entropy is an anthropomorphic concept in the sense that in a physical system correspond many thermodynamic systems. The physical system can be examined from many points of view each time examining…
It is known that statistical model selection as well as identification of dynamical equations from available data are both very challenging tasks. Physical systems behave according to their underlying dynamical equations which, in turn, can…
The problem of how to properly quantify redundant information is an open question that has been the subject of much recent research. Redundant information refers to information about a target variable S that is common to two or more…
This paper is twofold. In the first part, we present a refinement of the R\'enyi Entropy Power Inequality (EPI) recently obtained in \cite{BM16}. The proof largely follows the approach in \cite{DCT91} of employing Young's convolution…
Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its…
Entropy and its various generalizations are important in many fields, including mathematical statistics, communication theory, physics and computer science, for characterizing the amount of information associated with a probability…
The Mutual Information (MI) is an often used measure of dependency between two random variables utilized in information theory, statistics and machine learning. Recently several MI estimators have been proposed that can achieve parametric…
A method based on orthogonal function series interpolation of the square root probability density to analyze higher dimensional scattered data is presented. The method is targeted for the use-case when the model and/or data are available…
Emergent behaviors are a defining feature of complex systems, yet their quantitative characterization remains an open challenge, as traditional classifications rely mainly on visual inspection of spatio-temporal patterns. In this Letter, we…