Related papers: Weighted entropy: basic inequalities
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…
In mathematics information is a number that measures uncertainty (entropy) based on a probabilistic distribution, often of an obscure origin. In real life language information is a datum, a statement, more precisely, a formula. But such a…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence…
In this article the issues are discussed with the Bayesian approach, least-square fits, and most-likely fits. Trying to counter these issues, a method, based on weighted confidence, is proposed for estimating probabilities and other…
In a previous paper, we introduced an axiomatic system for information thermodynamics, deriving an entropy function that includes both thermodynamic and information components. From this function we derived an entropic probability…
Previously proposed measures of entanglement, such as entanglement of formation and assistance, are shown to be special cases of the relative entropy of entanglement. The difference between these measures for an ensemble of mixed states is…
We introduce quantum weighted entropy in analogy to an earlier notion of (classical) weighted entropy and derive many of its properties. These include the subadditivity, concavity and strong subadditivity property of quantum weighted…
The properties of an alternative definition of quantum entropy, based on Wigner functions, are discussed. Such definition emerges naturally from the Wigner representation of quantum mechanics, and can easily quantify the amount of…
Measure of the weighted cumulative entropy about the predictability of failure time of a system have been introduced in [3]. Referring properties of doubly truncated (interval) cumulative residual and past entropy, several bounds and…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
A novel heuristic approach is proposed here for time series data analysis, dubbed Generalized weighted permutation entropy, which amalgamates and generalizes beyond their original scope two well established data analysis methods:…
We establish a variant of the log-Sobolev and transport-information inequalities for mixture distributions. If a probability measure $\pi$ can be decomposed into components that individually satisfy such inequalities, then any measure $\mu$…
This note contributes to the understanding of generalized entropy power inequalities. Our main goal is to construct a counter-example regarding monotonicity and entropy comparison of weighted sums of independent identically distributed…
In this paper we introduce a new generalisation of the relative Fisher Information for Markov jump processes on a finite or countable state space, and prove an inequality which connects this object with the relative entropy and a large…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
Contemporary predictive models are hard to interpret as their deep nets exploit numerous complex relations between input elements. This work suggests a theoretical framework for model interpretability by measuring the contribution of…
Using known entropic and information inequalities new inequalities for some classical polynomials are obtained. Examples of Jacobi and Legendre polynomials are considered.