Related papers: Information Theory: A Tutorial Introduction
We generalize the Shannon's information theory in a nonadditive way by focusing on the source coding theorem. The nonadditive information content we adopted is consistent with the concept of the form invariance structure of the nonextensive…
The generalization of Shannon's theory to include messages with given autocorrelations is presented. The analytical calculation of the channel capacity is based on the transfer matrix method of the effective 1D Hamiltonian. This bridge…
We survey the prospects for an Information Dynamics which can serve as the basis for a fundamental theory of information, incorporating qualitative and structural as well as quantitative aspects. We motivate our discussion with some basic…
One major function of social networks (e.g., massive online social networks) is the dissemination of information, such as scientific knowledge, news, and rumors. Information can be propagated by the users of the network via natural…
The general idea of information entropy provided by C.E. Shannon "hangs over everything we do" and can be applied to a great variety of problems once the connection between a distribution and the quantities of interest is found. The Shannon…
From a traditional point of view, the value of information does not change during transmission. The Shannon information theory considers information transmission as a statistical phenomenon for measuring the communication channel capacity.…
The participation coefficient is a widely used metric of the diversity of a node's connections with respect to a modular partition of a network. An information-theoretic formulation of this concept of connection diversity, referred to here…
While Shannon's mutual information has widespread applications in many disciplines, for practical applications it is often difficult to calculate its value accurately for high-dimensional variables because of the curse of dimensionality.…
Semantic communication is regarded as the breakthrough beyond the Shannon paradigm, which transmits only semantic information to significantly improve communication efficiency. This article introduces a framework for generalized semantic…
Estimating the Shannon information associated with individual neurons is a non-trivial problem. Three key methods used to estimate the mutual information between neuron inputs and outputs are described, and a list of further readings is…
Shannon's channel coding theorem characterizes the maximal rate of information that can be reliably transmitted over a communication channel when optimal encoding and decoding strategies are used. In many scenarios, however, practical…
Information theory plays a central role in establishing fundamental limits on what any learning or estimation algorithm can -- and cannot -- achieve, regardless of computational power. In this chapter, we provide an introduction to these…
The Quantum Reverse Shannon Theorem states that any quantum channel can be simulated by an unlimited amount of shared entanglement and an amount of classical communication equal to the channel's entanglement assisted classical capacity. In…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
Shannon entropy is the most crucial foundation of Information Theory, which has been proven to be effective in many fields such as communications. Renyi entropy and Chernoff information are other two popular measures of information with…
Each generation of cellular networks is characterized by its distinct capabilities and innovations, which reflect the significant milestones reached with each new release. 5G has made substantial progress through the deployment of advanced…
Organized knowledge production can then be considered as the codification of communication. Communications leave traces that can be studied as indicators. Institutions can be considered as retention mechanisms functional for the…
We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of…
Similar to the energy flowing process in traditional heat engines, information could be considered to flow in the communication systems with the form of energy and entropy. Combining the thermodynamic Carnot machine and the classical…
Shannon's notion of relative information between two physical systems can function as foundation for statistical mechanics and quantum mechanics, without referring to subjectivism or idealism. It can also represent a key missing element in…