Related papers: An Elementary Proof of a Classical Information-The…
We take an information theoretic perspective on a classical sparse-sampling noisy linear model and present an analytical expression for the mutual information, which plays central role in a variety of communications/processing problems.…
Shannon's information theory deliberately excludes message semantics. This paper develops a rigorous framework for semantic communication that integrates formal proof systems with Shannon-theoretic tools. We introduce an axiomatic…
We investigate the fundamental limits of reliable communication over multi-view channels, in which the channel output is comprised of a large number of independent noisy views of a transmitted symbol. We consider first the setting of…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
We characterize mutual information as the unique map on ordered pairs of random variables satisfying a set of axioms similar to those of Faddeev's characterization of the Shannon entropy. There is a new axiom in our characterization however…
Consider a random non-centered multiple antenna radio transmission channel. Assume that the deterministic part of the channel is itself frequency selective, and that the random multipath part is represented by an ergodic stationary vector…
We develop a general formalism for representing and understanding structure in complex systems. In our view, structure is the totality of relationships among a system's components, and these relationships can be quantified using information…
Information Theory concepts and methodologies conform the background of how communication systems are studied and understood. They are mainly focused on the source-channel-receiver problem and on the asymptotic limits of accuracy and…
Muroga [M52] showed how to express the Shannon channel capacity of a discrete channel with noise [S49] as an explicit function of the transition probabilities. His method accommodates channels with any finite number of input symbols, any…
We derive the Einstein tensor from the Fisher information metric that is defined by the probability distribution of a statistical mechanical system. We find that the tensor naturally contains essential information of the energy-momentum…
Information theory has provided foundations for the theories of several application areas critical for modern society, including communications, computer storage, and AI. A key aspect of Shannon's 1948 theory is a sharp lower bound on the…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
Shannon mutual information provides a measure of how much information is, on average, contained in a set of neural activities about a set of stimuli. It has been extensively used to study neural coding in different brain areas. To apply a…
We explore a few common models on how correlations affect information. The main model considered is the Shannon mutual information $I(S:R_1,\cdots, R_i)$ over distributions with marginals $P_{S,R_i}$ fixed for each $i$, with the analogy in…
Most coding theorems in quantum Shannon theory can be proven using the decoupling technique: to send data through a channel, one guarantees that the environment gets no information about it; Uhlmann's theorem then ensures that the receiver…
An information theory description of finite systems explicitly evolving in time is presented for classical as well as quantum mechanics. We impose a variational principle on the Shannon entropy at a given time while the constraints are set…
In this paper, firstly, the Shannon channel capacity formula is briefly stated, and the relationship between the formula and the signal uncertainty principle is analyzed in order to prepare for deriving the formula which is able to break…
A communication theory for a transmitter broadcasting to many receivers is presented. In this case energetic considerations cannot be neglected as in Shannon theory. It is shown that, when energy is assigned to the information bit,…
We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories:…
Passive environment assisted communication takes place via a quantum channel modeled as a unitary interaction between the information carrying system and an environment, where the latter is controlled by a passive helper, who can set its…