相关论文: Shannon Shakes Hands with Chernoff: Big Data Viewp…
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual…
We show that the way in which the Shannon entropy of sequences produced by an information source converges to the source's entropy rate can be used to monitor how an intelligent agent builds and effectively uses a predictive model of its…
We present a problem relating measurements and information theory in spin foam models. In the three dimensional case of quantum gravity we can compute probabilities of spin network graphs and study the behaviour of the Shannon entropy…
This paper considers an information bottleneck problem with the objective of obtaining a most informative representation of a hidden feature subject to a R\'enyi entropy complexity constraint. The optimal bottleneck trade-off between…
Rough speaking, information theory deals with data transmitted over a channel such as the internet. Modern information theory is generally considered to have been founded in 1948 by Shannon in his seminal paper, "A mathematical theory of…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
While information is ubiquitously generated, shared, and analyzed in a modern-day life, there is still some controversy around the ways to asses the amount and quality of information inside a noisy optical channel. A number of theoretical…
In quantum Shannon theory, the way information is encoded and decoded takes advantage of the laws of quantum mechanics, while the way communication channels are interlinked is assumed to be classical. In this Letter we relax the assumption…
Shannon information theory is established based on probability and bits, and the communication technology based on this theory realizes the information age. The original goal of Shannon's information theory is to describe and transmit…
We explore a few common models on how correlations affect information. The main model considered is the Shannon mutual information $I(S:R_1,\cdots, R_i)$ over distributions with marginals $P_{S,R_i}$ fixed for each $i$, with the analogy in…
Subtraction of aligned images is a means to assess changes in a wide variety of clinical applications. In this paper we explore the information theoretical origin of Mutual Information (MI), which is based on Shannon's entropy.However, the…
Shannon entropy ($S$), R{\'e}nyi entropy ($R$), Tsallis entropy ($T$), Fisher information ($I$) and Onicescu energy ($E$) have been explored extensively in both \emph{free} H atom (FHA) and \emph{confined} H atom (CHA). For a given quantum…
In the past over two decades, very fruitful results have been obtained in information theory in the study of the Shannon entropy. This study has led to the discovery of a new class of constraints on the Shannon entropy called…
So far, feedback-driven systems have been discussed using (i) measurement and control, (ii) a tape interacting with a system or (iii) by identifying an implicit Maxwell demon in steady state transport. We derive the corresponding second…
Information and uncertainty are closely related and extensively studied concepts in a number of scientific disciplines such as communication theory, probability theory, and statistics. Increasing the information arguably reduces the…
Bounds on information combining are a fundamental tool in coding theory, in particular when analyzing polar codes and belief propagation. They usually bound the evolution of random variables with respect to their Shannon entropy. In recent…
Paleoclimate records are extremely rich sources of information about the past history of the Earth system. Information theory, the branch of mathematics capable of quantifying the degree to which the present is informed by the past,…
A direct connection of information entropy $S$ and kinetic energy $T$ is obtained for nuclei and atomic clusters, which establishes $T$ as a measure of the information in a distribution. It is conjectured that this is a universal property…
One of the main problems in cryptography is to give criteria to provide good comparators of cipher systems. The security of a cipher system must include the security of the algorithm, the security of the key generator and management module…