Related papers: Foundations of information theory for coding theor…
The information entropy budget and the rate of information transfer between variables is studied in the context of a nonlinear reduced-order atmospheric model. The key ingredients of the dynamics are present in this model, namely the…
We develop a general formalism for representing and understanding structure in complex systems. In our view, structure is the totality of relationships among a system's components, and these relationships can be quantified using information…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
In this paper, we consider a recently-proposed model of teaching and learning under uncertainty, in which a teacher receives independent observations of a single bit corrupted by binary symmetric noise, and sequentially transmits to a…
Shannon's classical information theory uses probability theory to analyze channels as mechanisms for information flow. In this paper, we generalize results of Martin, Allwein and Moskowitz for binary channels to show how some more modern…
Near-term quantum communication protocols suffer inevitably from channel noises, whose alleviation has been mostly attempted with resources such as multiparty entanglement or sophisticated experimental techniques. Generation of multiparty…
Information theory establishes the fundamental limits on data transmission, storage, and processing. Quantum information theory unites information theoretic ideas with an accurate quantum-mechanical description of reality to give a more…
We survey and introduce concepts and tools located at the intersection of information theory and network biology. We show that Shannon's information entropy, compressibility and algorithmic complexity quantify different local and global…
Information theory plays a central role in establishing fundamental limits on what any learning or estimation algorithm can -- and cannot -- achieve, regardless of computational power. In this chapter, we provide an introduction to these…
We give a non-technical introduction of the basic concepts of Quantum Information Theory along the distinction between possible and impossible machines. We then proceed to describe the mathematical framework of Quantum Information Theory.…
With the advent of massive data outputs at a regular rate, admittedly, signal processing technology plays an increasingly key role. Nowadays, signals are not merely restricted to physical sources, they have been extended to digital sources…
Shannon information theory is established based on probability and bits, and the communication technology based on this theory realizes the information age. The original goal of Shannon's information theory is to describe and transmit…
The general subject considered in this thesis is a recently discovered coding technique, polar coding, which is used to construct a class of error correction codes with unique properties. In his ground-breaking work, Ar{\i}kan proved that…
We propose an information transmission scheme by a swarm of anonymous oblivious mobile robots on a graph. The swarm of robots travel from a sender vertex to a receiver vertex to transmit a symbol generated at the sender. The codeword for a…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
We study reliable communication in uncoordinated vehicular communication from the perspective of Shannon theory. Our system model for the information transmission is that of an Arbitrarily Varying Channel (AVC): One sender-receiver pair…
Bounds on information combining are a fundamental tool in coding theory, in particular when analyzing polar codes and belief propagation. They usually bound the evolution of random variables with respect to their Shannon entropy. In recent…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
This paper introduces several fundamental concepts in information theory from the perspective of their origins in engineering. Understanding such concepts is important in neuroscience for two reasons. Simply applying formulae from…
We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined."…