Related papers: Information Bottleneck for an Oblivious Relay with…
Information velocity (IV) is a recently proposed notion to capture the speed of reliable information dissemination over a large-scale network. It is the speed at which reliable end-to-end communication over $k$ hops can be achieved within…
The Information Bottleneck (IB) framework is a general characterization of optimal representations obtained using a principled approach for balancing accuracy and complexity. Here we present a new framework, the Dual Information Bottleneck…
Deep neural networks suffer from poor generalization to unseen environments when the underlying data distribution is different from that in the training set. By learning minimum sufficient representations from training data, the information…
A network consisting of $n$ source-destination pairs and $m$ relays is considered. Focusing on the large system limit (large $n$), the throughput scaling laws of two-hop relaying protocols are studied for Rayleigh fading channels. It is…
The Information Bottleneck (IB) method (\cite{tishby2000information}) provides an insightful and principled approach for balancing compression and prediction for representation learning. The IB objective $I(X;Z)-\beta I(Y;Z)$ employs a…
In this paper, a two-hop network in which information is transmitted from a source via a relay to a destination is considered. It is assumed that the channels are static fading with additive white Gaussian noise. All nodes are equipped with…
The information bottleneck (IB) method seeks a compressed representation of data that preserves information relevant to a target variable for prediction while discarding irrelevant information from the original data. In its classical…
The information bottleneck (IB) method is a technique for extracting information that is relevant for predicting the target random variable from the source random variable, which is typically implemented by optimizing the IB Lagrangian that…
The Information Bottleneck (IB) method is an information theoretical framework to design a parsimonious and tunable feature-extraction mechanism, such that the extracted features are maximally relevant to a specific learning or inference…
Reinforcement Learning (RL) has recently achieved remarkable success in robotic control. However, most works in RL operate in simulated environments where privileged knowledge (e.g., dynamics, surroundings, terrains) is readily available.…
In this paper, we investigate the capacity of known interference channel, where the receiver knows the interference data but not the channel gain of the interference data. We first derive a tight upper bound for the capacity of this…
The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations $T$ of some random variable $X$ for the task of predicting $Y$. It is defined as a constrained optimization problem which maximizes…
We consider the problem of minimizing upper bounds and maximizing lower bounds on information rates of stationary and ergodic discrete-time channels with memory. The channels we consider can have a finite number of states, such as partial…
Multi-task learning (MTL) is an important subject in machine learning and artificial intelligence. Its applications to computer vision, signal processing, and speech recognition are ubiquitous. Although this subject has attracted…
Bayesian Inference and Information Bottleneck are the two most popular objectives for neural networks, but they can be optimised only via a variational lower bound: the Variational Information Bottleneck (VIB). In this manuscript we show…
We investigate the transmission of a secret message from Alice to Bob in the presence of an eavesdropper (Eve) and many of decode-and-forward relay nodes. Each link comprises a set of parallel channels, modeling for example an orthogonal…
With the advancement of Artificial Intelligence (AI) technology, next-generation wireless communication network is facing unprecedented challenge. Semantic communication has become a novel solution to address such challenges, with enhancing…
Large language models (LLMs) have recently demonstrated remarkable progress in reasoning capabilities through reinforcement learning with verifiable rewards (RLVR). By leveraging simple rule-based rewards, RL effectively incentivizes LLMs…
This paper investigates the maximum coding rate at which data can be transmitted over a noncoherent, multiple-input, multiple-output (MIMO) Rayleigh block-fading channel using an error-correcting code of a given blocklength with a…
We consider the impact of incomplete information on incentives for node cooperation in parallel relay networks with one source node, one destination node, and multiple relay nodes. All nodes are selfish and strategic, interested in…