Related papers: Non-equilibrium Information Envelopes and the Capa…
Information theory establishes the fundamental limits on data transmission, storage, and processing. Quantum information theory unites information theoretic ideas with an accurate quantum-mechanical description of reality to give a more…
Masking-based post-hoc explanation methods, such as KernelSHAP and LIME, estimate local feature importance by querying a black-box model under randomized perturbations. This paper formulates this procedure as communication over a query…
This chapter provides a comprehensive and self-contained discussion of the most recent developments of information theory of networks. Maximum entropy models of networks are the least biased ensembles enforcing a set of constraints and are…
Information inequalities appear in many database applications such as query output size bounds, query containment, and implication between data dependencies. Recently Khamis et al. proposed to study the algorithmic aspects of information…
We introduce \emph{Information Topology}: a framework that unifies information theory and algebraic topology by treating \emph{cycle closure} as the primitive operation of inference. The starting point is the \emph{dot-cycle dichotomy},…
Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
The paper is focused on the tradeoff between performance and decoding complexity per iteration for LDPC codes in terms of their gap (in rate) to capacity. The study of this tradeoff is done via information-theoretic bounds which also enable…
We introduce the use of entanglement entropy as a tool for studying the amount of information shared between the nodes of quantum complex networks. By considering the ground state of a network of coupled quantum harmonic oscillators, we…
The information entropy budget and the rate of information transfer between variables is studied in the context of a nonlinear reduced-order atmospheric model. The key ingredients of the dynamics are present in this model, namely the…
Existing information-theoretic frameworks based on maximum entropy network ensembles are not able to explain the emergence of heterogeneity in complex networks. Here, we fill this gap of knowledge by developing a classical framework for…
Superdense Coding is a cornerstone in secure quantum communication, exploiting pre-shared entanglement to encode two classical bits within a single qubit. However, noise and decoherence deteriorate entanglement quality, restricting both…
In this paper we consider an interacting two-agent sequential decision-making problem consisting of a Markov source process, a causal encoder with feedback, and a causal decoder. Motivated by a desire to foster links between control and…
Learning, prediction, and compression are intimately connected: a model that accurately predicts the next symbol in a sequence can be coupled with a source coder to compress that sequence near its information-theoretic limit. When tokenized…
Coding theorems and (strong) converses for memoryless quantum communication channels and quantum sources are proved: for the quantum source the coding theorem is reviewed, and the strong converse proven. For classical information…
The universal typical-signal estimators of entropy and cross entropy based on the asymptotics of recurrence and waiting times play an important role in information theory. Building on their construction, we introduce and study universal…
Quantum information theory studies the fundamental limits that physical laws impose on information processing tasks such as data compression and data transmission on noisy channels. This thesis presents general techniques that allow one to…
We consider a queue-channel model that captures the waiting time-dependent degradation of information bits as they wait to be transmitted. Such a scenario arises naturally in quantum communications, where quantum bits tend to decohere…
One of the primary goals of information theory is to provide limits on the amount of information it is possible to send through various types of communication channels, and to understand the encoding methods that will allow one to achieve…
We present novel data-processing inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal mappings, but are allowed to be…