Related papers: Question on Conditional Entropy
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
The microscopic explanation of entropy has been challenged from both experimental and theoretical point of view. The expression of entropy is derived from the first law of thermodynamics indicating that entropy or the second law of…
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of…
The expression for entropy sometimes appears mysterious - as it often is asserted without justification. This short manuscript contains a discussion of the underlying assumptions behind entropy as well as simple derivation of this…
A colloquial interpretation of entropy is that it is the knowledge gained upon learning the outcome of a random experiment. Conditional entropy is then interpreted as the knowledge gained upon learning the outcome of one random experiment…
Informational entropy is often identified as physical entropy. This is surprising because the two quantities are differently defined and furthermore the former is a subjective quantity while the latter is an objective one. We describe the…
The definition of nonequilibrium entropy is provided for the general nonequilibrium processes by connecting thermodynamics with statistical physics, and the principle of entropy increment in the nonequilibrium processes is also proved in…
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…
A notion of entropy is introduced for causal fermion systems. This entropy is a measure of the state of disorder of a causal fermion system at a given time compared to the vacuum. The definition is given both in the finite and…
The product expansion of conditional probabilities for belief nets is not maximum entropy. This appears to deny a desirable kind of assurance for the model. However, a kind of guarantee that is almost as strong as maximum entropy can be…
In this paper a general definition of quantum conditional entropy for infinite-dimensional systems is given based on recent work of Holevo and Shirokov arXiv:1004.2495 devoted to quantum mutual and coherent informations in the…
We define and investigate a notion of entropy for quantum error correcting codes. The entropy of a code for a given quantum channel has a number of equivalent realisations, such as through the coefficients associated with the Knill-Laflamme…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
The routine definitions of both entropy, and differential entropy show inconsistencies that make them not reciprocally coherent. We propose a few possible modifications of these quantities so that 1) they no longer show incongruities, 2)…
Quantum uncertainty relations are formulated in terms of relative entropy between distributions of measurement outcomes and suitable reference distributions with maximum entropy. This type of entropic uncertainty relation can be applied…
The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning. While the properties of these measures, which are rooted…
Entropy is a very useful concept from physics that tries to explain how a system behaves from a point of view of the thermodynamics. However, there are two ways to explain entropy, and it depends on if we are studying a microsystem or a…
In this paper, the concept of L-algebra is revisited and after that, the article is prepared to deal with the notion of the entropy of an L-algebra. If a set has an L-algebraic structure, it is possible to calculate the degree of…
Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to…
Entropy is a concept that has traditionally been reliant on a definite notion of causality. However, without a definite notion of causality, the concept of entropy is not all lost. Indefinite causal structure results from combining…