Related papers: Directed Information: Estimation, Optimization and…
The cognitive interference channel is an interference channel in which one transmitter is non-causally provided with the message of the other transmitter. This channel model has been extensively studied in the past years and capacity…
An upper bound on the feedback capacity of unifilar finite-state channels (FSCs) is derived. A new technique, called the $Q$-contexts, is based on a construction of a directed graph that is used to quantize recursively the receiver's output…
Direct Preference Optimization (DPO) has been widely used for aligning language models with human preferences in a supervised manner. However, several key questions remain unresolved: the rationale behind its log-ratio reward, how the…
The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
Communication systems are usually designed by assuming perfect channel state information (CSI). However, in many practical scenarios, only a noisy estimate of the channel is available, which may strongly differ from the true channel. This…
Deep Neural Networks (DNNs) are often examined at the level of their response to input, such as analyzing the mutual information between nodes and data sets. Yet DNNs can also be examined at the level of causation, exploring "what does…
We extend the Blahut-Arimoto algorithm for maximizing Massey's directed information. The algorithm can be used for estimating the capacity of channels with delayed feedback, where the feedback is a deterministic function of the output. In…
We develop a theoretical framework for defining and identifying flows of information in computational systems. Here, a computational system is assumed to be a directed graph, with "clocked" nodes that send transmissions to each other along…
Artificial intelligence (AI) has emerged as a promising tool for channel state information (CSI) feedback. While recent research primarily focuses on improving feedback accuracy on a specific dataset through novel architectures, the…
Mutual Information is the metric that is used to perform link adaptation, which allows to achieve rates near capacity. The computation of adaptive transmission modes is achieved by employing the mapping between the Signal to Noise Ratio and…
We present several novel identities and inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal mappings, but are allowed…
Fundamental limitations or performance trade-offs/limits are important properties and constraints of both control and filtering systems. Among various trade-off metrics, total information rate that characterizes the sensitivity trade-offs…
A communication setup is considered where a transmitter wishes to convey a message to a receiver and simultaneously estimates the state of that receiver through a common waveform. The state is estimated at the transmitter by means of…
Age of Incorrect Information (AoII) is a newly introduced performance metric that considers communication goals. Therefore, comparing with traditional performance metrics and the recently introduced metric - Age of Information (AoI), AoII…
Finite-time optimal feedback control for flow networks under information constraints is studied. By utilizing the framework of multi-parametric linear programming, it is demonstrated that when cost/constraints can be modeled or approximated…
We propose a new measure to estimate the direction of information flux in multivariate time series from complex systems. This measure, based on the slope of the phase spectrum (Phase Slope Index) has invariance properties that are important…
Mutual information (MI)-based guidelines have recently proven to be effective for designing task-oriented communication systems, where the ultimate goal is to extract and transmit task-relevant information for downstream task. This paper…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
The problem of state communication over a discrete memoryless channel with discrete memoryless state is studied when the state information is available strictly causally at the encoder. It is shown that block Markov encoding, in which the…