Related papers: Towards a Semantic Information Theory (Introducing…
Information theory has provided foundations for the theories of several application areas critical for modern society, including communications, computer storage, and AI. A key aspect of Shannon's 1948 theory is a sharp lower bound on the…
This article consists of a very short introduction to classical and quantum information theory. Basic properties of the classical Shannon entropy and the quantum von Neumann entropy are described, along with related concepts such as…
We generalize the Shannon's information theory in a nonadditive way by focusing on the source coding theorem. The nonadditive information content we adopted is consistent with the concept of the form invariance structure of the nonextensive…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
This article serves as a brief introduction to the Shannon information theory. Concepts of information, Shannon entropy and channel capacity are mainly covered. All these concepts are developed in a totally combinatorial flavor. Some issues…
Claude Shannons information theory (1949) has had a revolutionary impact on communication science. A crucial property of his framework is that it decouples the meaning of a message from the mechanistic details from the actual communication…
Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by…
Information theory is a statistical theory dealing with the relative state of detectors and physical systems. Because of this physicality of information, the classical framework of Shannon needs to be extended to deal with quantum…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
Logical information theory is the quantitative version of the logic of partitions just as logical probability theory is the quantitative version of the dual Boolean logic of subsets. The resulting notion of information is about…
This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…
Information theory is a statistical theory concerned with the relative state of detectors and physical systems. As a consequence, the classical framework of Shannon needs to be extended to deal with quantum detectors, possibly moving at…
Information theory, introduced by Shannon, has been extremely successful and influential as a mathematical theory of communication. Shannon's notion of information does not consider the meaning of the messages being communicated but only…
One of the key aspects of Shannon's theory is that it provides guidance for designing the most efficient systems, such as minimizing errors and clarifying the limits of coding. Such theories have made great developments in the 50 years…
We present a theory for modeling the structure of information and a language (Quanta) expressing the theory. Unlike Shannon's information theory, which focuses on the amount of information in an information system, we focus on the structure…
Biologists rely heavily on the language of information, coding, and transmission that is commonplace in the field of information theory as developed by Claude Shannon, but there is open debate about whether such language is anything more…
Quantum error-correcting codes are analyzed from an information-theoretic perspective centered on quantum conditional and mutual entropies. This approach parallels the description of classical error correction in Shannon theory, while…
Shannon information theory is established based on probability and bits, and the communication technology based on this theory realizes the information age. The original goal of Shannon's information theory is to describe and transmit…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…