Related papers: Questions, relevance and relative entropy
Probability distributions can be read as simple expressions of information. Each continuous probability distribution describes how information changes with magnitude. Once one learns to read a probability distribution as a measurement scale…
The quantum relative entropy is a fundamental quantity in quantum information science, characterizing the distinguishability between two quantum states. However, this quantity is not additive in general for correlated quantum states,…
Uncertainty relations give upper bounds on the accuracy by which the outcomes of two incompatible measurements can be predicted. While established uncertainty relations apply to cases where the predictions are based on purely classical data…
Entropy is a very useful concept from physics that tries to explain how a system behaves from a point of view of the thermodynamics. However, there are two ways to explain entropy, and it depends on if we are studying a microsystem or a…
The assumption that a complete description of an early state of the universe does not privilege any position or direction in space leads to a unified account of probability in cosmology, macroscopic physics, and quantum mechanics. Such a…
We introduce a method to measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natural language outputs of foundation models. We show that measuring uncertainty in…
Various measures have been suggested recently for quantifying the coherence of a quantum state with respect to a given basis. We first use two of these, the l_1-norm and relative entropy measures, to investigate tradeoffs between the…
We have presented first an axiomatic derivation of Boltzmann entropy on the basis of two axioms consistent with two basic properties of thermodynamic entropy. We have then studied the relationship between Boltzmann entropy and information…
We show that the natural scaling of measurement for a particular problem defines the most likely probability distribution of observations taken from that measurement scale. Our approach extends the method of maximum entropy to use…
There has not been an established mathematical measure of evidence. Some Bayesians have argued that probability can be an objectively correct measure of ``rational degrees of belief,'' which we do not distinguish from evidence. However,…
An analysis of quantum measurement is presented that relies on an information-theoretic description of quantum entanglement. In a consistent quantum information theory of entanglement, entropies (uncertainties) conditional on measurement…
When the difference between changes in energy and entropy at a given temperature is correlated with the ratio between the same changes in energy and entropy at zero average free energy of an ensemble of similar but distinct molecule-sized…
We examine the relationship between the Bayesian and information-theoretic formulations of source separation algorithms. This work makes use of the relationship between the work of Claude E. Shannon and the "Recent Contributions" by Warren…
We show how to determine the maximum and minimum possible values of one measure of entropy for a given value of another measure of entropy. These maximum and minimum values are obtained for two standard forms of probability distribution (or…
The widely claimed replicability crisis in science may lead to revised standards of significance. The customary frequentist confidence intervals, calibrated through hypothetical repetitions of the experiment that is supposed to have…
Problem of classification of all the set of entangled states is considered. Invariance of entangled states relative to transformations from a group of symmetry of qubit space leads to classification of all states of the system through…
Unsolved controversies about uncertainty relations and quantum measurements still persists nowadays. They originate around the shortcomings regarding the conventional interpretation of uncertainty relations. Here we show that the respective…
The notion of entropy penetrates much of science. A key feature of the all-important notion of Boltzmann-Gibbs-Shannon entropy is its extensivity (additivity over independent subsystems). However, there is a need for other quantities. In…
This thesis develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
A common statistical situation concerns inferring an unknown distribution Q(x) from a known distribution P(y), where X (dimension n), and Y (dimension m) have a known functional relationship. Most commonly, n<m, and the task is relatively…