Related papers: Derangetropy in Probability Distributions and Info…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
Entanglement measures based on a logarithmic functional form naturally emerge in any attempt to quantify the degree of entanglement in the state of a multipartite quantum system. These measures can be regarded as generalizations of the…
In statistical analysis, quantifying uncertainties through measures such as entropy, extropy, varentropy, and varextropy is of fundamental importance for understanding distribution functions. This paper investigates several properties of…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
Graph based entropy, an index of the diversity of events in their distribution to parts of a co-occurrence graph, is proposed for detecting signs of structural changes in the data that are informative in explaining latent dynamics of…
Information functionals allow to quantify the degree of randomness of a given probability distribution, either absolutely (through min/max entropy principles) or relative to a prescribed reference one. Our primary aim is to analyze the…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
Entropy or information is a fundamental quantity contained in a system in statistical mechanics and information theory. In this paper, a definition of classical information entropy of parton distribution functions is suggested. The…
The histogram is a key method for visualizing data and estimating the underlying probability distribution. Incorrect conclusions about the data result from over or under-binning. A new method based on the Shannon entropy of the histogram…
We introduce a novel method, called Dispersion Entropy for Graph Signals, $DE_G$, as a powerful tool for analysing the irregularity of signals defined on graphs. We demonstrate the effectiveness of $DE_G$ in detecting changes in the…
In Reliability Theory, uncertainty is measured by the Shannon entropy. Recently, in order to analyze the variability of such measure, varentropy has been introduced and studied. In this paper we define a new concept of varentropy for past…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically,…
In this paper we develop the concept of information transfer between the Borel-measurable sets for a dynamical system described by a measurable space and a non-singular transformation. The concept is based on how Shannon entropy is…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
Representation learning is an approach that allows to discover and extract the factors of variation from the data. Intuitively, a representation is said to be disentangled if it separates the different factors of variation in a way that is…
Probability theory is fundamental for modeling uncertainty, with traditional probabilities being real and non-negative. Complex probability extends this concept by allowing complex-valued probabilities, opening new avenues for analysis in…