Related papers: Questions, relevance and relative entropy
Previously proposed measures of entanglement, such as entanglement of formation and assistance, are shown to be special cases of the relative entropy of entanglement. The difference between these measures for an ensemble of mixed states is…
People employ their knowledge to recognize things. This paper is concerned with how to measure people's knowledge for recognition and how it changes. The discussion is based on three assumptions. Firstly, we construct two evolution process…
Statistical mechanics relies on the complete though probabilistic description of a system in terms of all the microscopic variables. Its object is to derive therefrom static and dynamic properties involving some reduced set of variables.…
The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs be obtained from the data sample itself. We propose a semi-parametric…
In this paper we review various information-theoretic characterizations of the approach to equilibrium in biological systems. The replicator equation, evolutionary game theory, Markov processes and chemical reaction networks all describe…
We discuss how the method of maximum entropy, MaxEnt, can be extended beyond its original scope, as a rule to assign a probability distribution, to a full-fledged method for inductive inference. The main concept is the (relative) entropy…
We introduce a new information-theoretic formulation of quantum measurement uncertainty relations, based on the notion of relative entropy between measurement probabilities. In the case of a finite-dimensional system and for any approximate…
In this work, we propose a theory for information matching. It is motivated by the observation that retrieval is about the relevance matching between two sets of properties (features), namely, the information need representation and…
We consider a probability distribution depending on a real parameter $x$. As functions of $x$, the R\'enyi entropy and the Tsallis entropy can be expressed in terms of the associated index of coincidence $S(x)$. We establish recurrence…
I report the existence of exactly one non-trivial solution to the equation $i(A,B)+i(A,\neg B)+i(\neg A,B)+i(\neg A,\neg B)= 0$, where $i(A,B)=\log\frac{P(A\text{ and }B)}{P(A)P(B)}$, and $P(A)$ is the probability of the proposition $A$.…
Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…
Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease…
Although some information-theoretic measures of uncertainty or granularity have been proposed in rough set theory, these measures are only dependent on the underlying partition and the cardinality of the universe, independent of the lower…
We consider the relation between knowledge and certainty, where a fact is known if it is true at all worlds an agent considers possible and is certain if it holds with probability 1. We identify certainty with probabilistic belief. We show…
The peculiar uncertainty or randomness of quantum measurements stems from coherence, whose information-theoretic characterization is currently under investigation. Under the resource theory of coherence, it is interesting to investigate…
The usual conjectures of quantum measurements approaches, inspired from the traditional interpretation of Heisenberg's ("uncertainty") relations, are proved as being incorrect. A group of reconsidered conjectures and a corresponding new…
The form and justification of inductive inference rules depend strongly on the representation of uncertainty. This paper examines one generic representation, namely, incomplete information. The notion can be formalized by presuming that the…
Learning is a distinctive feature of intelligent behaviour. High-throughput experimental data and Big Data promise to open new windows on complex systems such as cells, the brain or our societies. Yet, the puzzling success of Artificial…
We take another look at the general problem of selecting a preferred probability measure among those that comply with some given constraints. The dominant role that entropy maximization has obtained in this context is questioned by arguing…
At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the…