相关论文: A genuinely natural information measure

Entropy is a key measure in studies related to information theory and its many applications. Campbell of the first time recognized that exponential of Shannons entropy is just the size of the sample space when the distribution is uniform.…

A cornerstone of our understanding of both biological and artificial neural networks is that they store information in the strengths of synaptic connections among the neurons. However, in contrast to the well-established theory for…

Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish…

The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…

We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…

Shannon's mathematical theory of communication defines fundamental limits on how much information can be transmitted between the different components of any man-made or biological system. This paper is an informal but rigorous introduction…

We carry out a systematic study of uncertainty measures that are generic to dynamical processes of varied origins, provided they induce suitable continuous probability distributions. The major technical tool are the information theory…

We propose an information theoretic model for sociological networks. The model is a microcanonical ensemble of states and particles. The states are the possible pairs of nodes (i.e. people, sites and alike) which exchange information. The…

Understanding the complexity of human language requires an appropriate analysis of the statistical distribution of words in texts. We consider the information retrieval problem of detecting and ranking the relevant words of a text by means…

A tight information-theoretic measurement uncertainty relation is experimentally tested with neutron spin-1/2 qubits. The noise associated to the measurement of an observable is defined via conditional Shannon entropies and a tradeoff…

We develop a complexity measure for large-scale economic systems based on Shannon's concept of entropy. By adopting Leontief's perspective of the production process as a circular flow, we formulate the process as a Markov chain. Then we…

Claude Shannon coined entropy to quantify the uncertainty of a random distribution for communication coding theory. We observe that the uncertainty nature of entropy also limits its direct usage in mathematical modeling. Therefore we…

We provide a simple physical interpretation, in the context of the second law of thermodynamics, to the information inequality (a.k.a. the Gibbs' inequality, which is also equivalent to the log-sum inequality), asserting that the relative…

Integrated information theory is a mathematical, quantifiable theory of conscious experience. The linchpin of this theory, the $\phi$ measure, quantifies a system's irreducibility to disjoint parts. Purely as a measure of irreducibility, we…

Given any finite set equipped with a probability measure, one may compute its Shannon entropy or information content. The entropy becomes the logarithm of the cardinality of the set when the uniform probability is used. Leinster introduced…

One of the key aspects of Shannon's theory is that it provides guidance for designing the most efficient systems, such as minimizing errors and clarifying the limits of coding. Such theories have made great developments in the 50 years…

Ranked set sampling is a sampling design which has a wide range of applications in industrial statistics, and environmental and ecological studies, etc.. It is well known that ranked set samples provide more Fisher information than simple…

We consider a natural measure of relevance: the reduction in optimal prediction risk in the presence of side information. For any given loss function, this relevance measure captures the benefit of side information for performing inference…

Information is a valuable asset for agents in socio-economic systems, a significant part of the information being entailed into the very network of connections between agents. The different interlinkages patterns that agents establish may,…

Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…