Related papers: Entropy and Source Coding for Integer-Dimensional …
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…
Partial differential equations are ubiquitous in almost all applications of mathematics, where they provide a natural mathematical description of many phenomena involving change in physical, chemical, biological, and social processes. The…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
The entropy of a graph is an information-theoretic quantity which expresses the complexity of a graph \cite{DM1,M}. After Shannon introduced the definition of entropy to information and communication, many generalizations of the entropy…
This paper serves a twofold purpose. First, a unified perspective on diversity indices is introduced based on an entropic basis. It is shown that the class of all linear combinations of the entropic basis, referred to as the class of linear…
Similarity-sensitive entropy measures the uncertainty of a probability law relative to a similarity kernel that encodes the distinguishability between states. We develop a measure-theoretic treatment covering both finite similarity matrices…
Shearer's inequality bounds the sum of joint entropies of random variables in terms of the total joint entropy. We give another lower bound for the same sum in terms of the individual entropies when the variables are functions of…
The aim of this note is to introduce a notion of dynamical entropy, which we call infinite-product entropy, for probability measures on (countable) infinite cartesian product of any measurable space with itself. The idea behind the…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
This study considers the unconditional smooth R\'{e}nyi entropy, the smooth conditional R\'{e}nyi entropy proposed by Kuzuoka [\emph{IEEE Trans.\ Inf.\ Theory}, vol.~66, no.~3, pp.~1674--1690, 2020], and a new quantity which we term the…
This paper studies the behavior of the entropy numbers of classes of functions with bounded integral norms from a given finite dimensional linear subspace. Upper bounds of these entropy numbers in the uniform norm are obtained and applied…
Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual…
The diversity of the symbols of the information source is calculated following the definition that entropy is the information loss and following a new entropy-symbol similarity relation after the rejection of the Gibbs paradox statement.…
The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…
We show that Frenkel's integral representation of the quantum relative entropy provides a natural framework to derive continuity bounds for quantum information measures. Our main general result is a dimension-independent semi-continuity…
In classical physics, entropy quantifies the randomness of large systems, where the complete specification of the state, though possible in theory, is not possible in practice. In quantum physics, despite its inherently probabilistic…
Information-theoretic quantities like entropy and mutual information have found numerous uses in machine learning. It is well known that there is a strong connection between these entropic quantities and submodularity since entropy over a…
A surrogate data analysis is presented, which is based on the fluctuations of the ``entropy'' $S$ defined in the natural time-domain [Phys. Rev. E {\bf 68}, 031106, 2003]. This entropy is not a static one as, for example, the Shannon…
We study an information analogue of infinitely divisible probability distributions, where the i.i.d. sum is replaced by the joint distribution of an i.i.d. sequence. A random variable $X$ is called informationally infinitely divisible if,…
Entropic uncertainty relations for the position and momentum within the generalized uncertainty principle are examined. Studies of this principle are motivated by the existence of a minimal observable length. Then the position and momentum…