Related papers: Measuring questions: relevance and its relation to…
A logic is defined that allows to express information about statistical probabilities and about degrees of belief in specific propositions. By interpreting the two types of probabilities in one common probability space, the semantics given…
We discuss how the method of maximum entropy, MaxEnt, can be extended beyond its original scope, as a rule to assign a probability distribution, to a full-fledged method for inductive inference. The main concept is the (relative) entropy…
A central question for knowledge representation is how to encode and handle uncertain knowledge adequately. We introduce the probabilistic description logic ALCP that is designed for representing context-dependent knowledge, where the…
We study analytically the corrections to the leading terms in the Renyi entropy of a massive lattice theory, showing significant deviations from naive expectations. In particular, we show that finite size and finite mass effects give rise…
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
The ``Gibbs Paradox'' refers to several related questions concerning entropy in thermodynamics and statistical mechanics: whether it is an extensive quantity or not, how it changes when identical particles are mixed, and the proper way to…
We study the correlations of classical and quantum systems from the information theoretical points of view. We analyze a simple measure of correlations based on entropy (such measure was already investigated as the degree of entanglement by…
We propose a definition for the entanglement entropy of a gauge theory on a spatial lattice. Our definition applies to any subset of links in the lattice, and is valid for both Abelian and Non-Abelian gauge theories. For $\mathbb{Z}_N$ and…
Rooted trees with probabilities are used to analyze properties of a variable length code. A bound is derived on the difference between the entropy rates of the code and a memoryless source. The bound is in terms of normalized informational…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
Entropy plays a crucial role in characterization of information and entanglement, but it is not a scalar quantity and for many systems it is different for different relativistic observers. Loop quantum gravity predicts the…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…
Large language models (LLMs) sometimes demonstrate poor performance on knowledge-intensive tasks, commonsense reasoning is one of them. Researchers typically address these issues by retrieving related knowledge from knowledge graphs or…
We consider some questions concerning the monotonicity properties of entropy and mean entropy of states on translationally invariant systems (classical lattice, quantum lattice and quantum continuous). By taking the property of strong…
The problem addressed concerns the determination of the average number of successive attempts of guessing a word of a certain length consisting of letters with given probabilities of occurrence. Both first- and second-order approximations…
Entanglement entropy is a valuable tool for characterizing the correlation structure of quantum field theories. When applied to gauge theories, subtleties arise which prevent the factorization of the Hilbert space underlying the notion of…
Entropy is a quantity which is of great importance in physics and chemistry. The concept comes out of thermodynamics, proposed by Rudolf Clausius in his analysis of Carnot cycle and linked by Ludwig Boltzmann to the number of specific ways…
Stimulated by the need of describing useful notions related to information measures, we introduce the `pdf-related distributions'. These are defined in terms of transformation of absolutely continuous random variables through their own…
The relative entropy of a correlated state and an uncorrelated reference state is a reasonable measure for the degree of correlations. A key question is however which uncorrelated state to compare to. The relative entropy becomes minimal…