Related papers: Unnormalized Measures in Information Theory
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
In physics we attempt to infer the rules governing a system given only the results of imprecise measurements. This is an ill-posed problem because certain features of the system's state cannot be resolved by the measurements. However, by…
In the Bayesian approach to probability theory, probability quantifies a degree of belief for a single trial, without any a priori connection to limiting frequencies. In this paper we show that, despite being prescribed by a fundamental…
Although there is growing interest in measuring integrated information in computational and cognitive systems, current methods for doing so in practice are computationally unfeasible. Existing and novel integration measures are investigated…
No consensus seems to exist as to what constitutes a measurement which is still considered somewhat mysterious in many respects in quantum mechanics. At successive stages mathematical theory of measure, metrology and measurement theory…
We introduce a new mathematical framework for the probabilistic description of an experiment on a system of any type in terms of information representing this system initially. Based on the notions of an information state and a generalized…
Conformal Prediction (CP) is a distribution-free uncertainty estimation framework that constructs prediction sets guaranteed to contain the true answer with a user-specified probability. Intuitively, the size of the prediction set encodes a…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…
Quantification starts with sum and product rules that express combination and partition. These rules rest on elementary symmetries that have wide applicability, which explains why arithmetical adding up and splitting into proportions are…
Information diagram and the I-measure are useful mnemonics where random variables are treated as sets, and entropy and mutual information are treated as a signed measure. Although the I-measure has been successful in machine proofs of…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
A basic information theoretic model for summarization is formulated. Here summarization is considered as the process of taking a report of $v$ binary objects, and producing from it a $j$ element subset that captures most of the important…
In the first part of this two-part article, we have introduced and analyzed a multidimensional model, called the 'general tension-reduction' (GTR) model, able to describe general quantum-like measurements with an arbitrary number of…
Judging the significance and reproducibility of quantitative research requires a good understanding of relevant uncertainties, but it is often unclear how well these have been evaluated and what they imply. Reported scientific uncertainties…
The usual conjectures of quantum measurements approaches, inspired from the traditional interpretation of Heisenberg's ("uncertainty") relations, are proved as being incorrect. A group of reconsidered conjectures and a corresponding new…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
Integrated information theory is a mathematical, quantifiable theory of conscious experience. The linchpin of this theory, the $\phi$ measure, quantifies a system's irreducibility to disjoint parts. Purely as a measure of irreducibility, we…
Pimentel et al. (2020) recently analysed probing from an information-theoretic perspective. They argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
It is well understood that Bayesian decision theory and average case analysis are essentially identical. However, if one is interested in performing uncertainty quantification for a numerical task, it can be argued that standard approaches…