Related papers: Directed Data-Processing Inequalities for Systems …
Pairwise interactions between individuals are taken as fundamental drivers of collective behavior responsible for group cohesion and decision-making. While an individual directly influences only a few neighbors, over time indirect…
We prove a data processing inequality for quantum communication channels, which states that processing a received quantum state may never increase the mutual information between input and output states.
Meta-learning automatically infers an inductive bias by observing data from a number of related tasks. The inductive bias is encoded by hyperparameters that determine aspects of the model class or training algorithm, such as initialization…
We develop a theoretical framework for defining and identifying flows of information in computational systems. Here, a computational system is assumed to be a directed graph, with "clocked" nodes that send transmissions to each other along…
We consider a coded distributed computing problem in a ring-based communication network, where $N$ computing nodes are arranged in a ring topology and each node can only communicate with its neighbors within a constant distance $d$. To…
Fundamental limits on the controllability of physical systems are discussed in the light of information theory. It is shown that the second law of thermodynamics, when generalized to include information, sets absolute limits to the minimum…
Time-triggered and event-triggered control strategies for stabilization of an unstable plant over a rate-limited communication channel subject to unknown, bounded delay are studied and compared. Event triggering carries implicit…
Learning effective netlist representations is fundamentally constrained by the scarcity of labeled datasets, as real designs are protected by Intellectual Property (IP) and costly to annotate. Existing work therefore focuses on small-scale…
This paper presents generalized channel coding theorems for a time-slotted distributed communication system where a transmitter-receiver pair is communicating in parallel with other transmitters. Assume that the channel code of each…
Information transfer between coupled stochastic dynamics, measured by transfer entropy and information flow, is suggested as a physical process underlying the causal relation of systems. While information transfer analysis has booming…
In this study we consider rateless coding over discrete memoryless channels (DMC) with feedback. Unlike traditional fixed-rate codes, in rateless codes each codeword is infinitely long, and the decoding time depends on the confidence level…
In this paper, we consider a hierarchical distributed multi-task learning (MTL) system where distributed users wish to jointly learn different models orchestrated by a central server with the help of a layer of multiple relays. Since the…
Humans communicate using systems of interconnected stimuli or concepts -- from language and music to literature and science -- yet it remains unclear how, if at all, the structure of these networks supports the communication of information.…
The moderate deviation regime is concerned with the finite block length trade-off between communication cost and error for information processing tasks in the asymptotic regime, where the communication cost approaches a capacity-like…
Looped Transformers provide advantages in parameter efficiency, computational capabilities, and generalization for reasoning tasks. However, their expressive power regarding function approximation remains underexplored. In this paper, we…
Spin networks are endowed with an Information Transfer Fidelity (ITF), which defines an absolute upper bound on the probability of transmission of an excitation from one spin to another. The ITF is easily computable but the bound can be…
Information must take up space, must weigh, and its flux must be limited. Quantum limits on communication and information storage leading to these conclusions are here described. Quantum channel capacity theory is reviewed for both steady…
We consider the on-time transmissions of a sequence of packets over a fading channel.Different from traditional in-time communications, we investigate how many packets can be received $\delta$-on-time, meaning that the packet is received…
In this article, we consider a nonlinear process with delayed dynamics to be controlled over a communication network in the presence of disturbances and study robustness of the resulting closed-loop system with respect to network-induced…
A methodology is developed to realized optimal channel input conditional distributions, which maximize the finite-time horizon directed information, for channels with memory and feedback, by information lossless randomized strategies. The…