Related papers: Weighted entropy: basic inequalities
Word embeddings capture semantic relationships based on contextual information and are the basis for a wide variety of natural language processing applications. Notably these relationships are solely learned from the data and subsequently…
Different notions of entropy play a fundamental role in the classical theory of dynamical systems. Unlike many other concepts used to analyze autonomous dynamics, both measure-theoretic and topological entropy can be extended quite…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
Conscious experience is awash with underlying relationships. Moreover, for various brain regions such as the visual cortex, the system is biased toward some states. Representing this bias using a probability distribution shows that the…
The object of this paper is to introduce a new weighted information generating function whose derivative at point 1 gives some well known measures of information. Some properties and particular cases of the proposed generating function have…
We establish analogues of the Bergstr\"om and Bonnesen inequalities, related to determinants and volumes respectively, for the entropy power and for the Fisher information. The obtained inequalities strengthen the well-known convolution…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
A new combinatorial-probabilistic diagnostic entropy has been introduced. It describes the pair-wise sum of probabilities of system conditions that have to be distinguished during the diagnosing process. The proposed measure describes the…
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality recently proposed by Pawlowski et. al. (arXiv:0905.2992). We consider two…
Feng--Huang (2016) introduced weighted topological entropy and pressure for factor maps between dynamical systems and established its variational principle. Tsukamoto (2022) redefined those invariants quite differently for the simplest case…
The notion of complex-valued information entropy measure is presented. It applies in particular to directed networks (digraphs). The corresponding statistical physics notions are outlined. The studied network, serving as a case study, in…
Importance sampling is a central idea underlying off-policy prediction in reinforcement learning. It provides a strategy for re-weighting samples from a distribution to obtain unbiased estimates under another distribution. However,…
Given an arbitrary continuous probability density function, it is introduced a conjugated probability density, which is defined through the Shannon information associated with its cumulative distribution function. These new densities are…
Although some information-theoretic measures of uncertainty or granularity have been proposed in rough set theory, these measures are only dependent on the underlying partition and the cardinality of the universe, independent of the lower…
Entropy is a central concept in physics, but can be challenging to calculate even for systems that are easily simulated. This is exacerbated out of equilibrium, where generally little is known about the distribution characterizing simulated…
We introduce an information order on experiments based on weighted garbling, a generalization of the standard notion of garbling. In this order, an experiment is more informative than another if the latter is a weighted garbling of the…
Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
An expression is proposed for determining the error caused on entropy estimates by finite sample effects. This expression is based on the Ansatz that the ranked distribution of probabilities tends to follow an empirical Zipf law.
Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to…