Related papers: Non-Stochastic Information Theory
Let $W$ be a binary-input memoryless symmetric (BMS) channel with Shannon capacity $I(W)$ and fix any $\alpha > 0$. We construct, for any sufficiently small $\delta > 0$, binary linear codes of block length $O(1/\delta^{2+\alpha})$ and rate…
In this paper, we formally define and analyze the class of noisy permutation channels. The noisy permutation channel model constitutes a standard discrete memoryless channel (DMC) followed by an independent random permutation that reorders…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…
Biological neural systems encode and transmit information as patterns of activity tracing complex trajectories in high-dimensional state-spaces, inspiring alternative paradigms of information processing. Heteroclinic networks, naturally…
We propose a new non-equilibrium model for spatial pattern formation on the basis of local information transfer. Unlike standard models of pattern formation it is not based on the Turing instability. Information is transmitted through the…
This work introduces capacity limits for molecular timing (MT) channels, where information is modulated on the release timing of small information particles, and decoded from the time of arrival at the receiver. It is shown that the random…
We consider the problem of communicating over a channel that breaks the message block into fragments of random lengths, shuffles them out of order, and deletes a random fraction of the fragments. Such a channel is motivated by applications…
A general theory for computing information transfers in nonlinear stochastic systems driven by deterministic forcing and additive and/or multiplicative noises, is presented. It extends the Liang-Kleeman framework of causality inference to…
Data shuffling is one of the fundamental building blocks for distributed learning algorithms, that increases the statistical gain for each step of the learning process. In each iteration, different shuffled data points are assigned by a…
Particularly motivated by DNA storage channels, we consider channels with synchronization errors modeled as insertions and deletions, along with substitutions. We focus on the case where the synchronization error process has memory and…
In this paper, we consider some long-standing problems in communication systems with access to noisy feedback. We introduce a new notion, the residual directed information, to capture the effective information flow (i.e. mutual information…
Classical multiuser information theory studies the fundamental limits of models with a fixed (often small) number of users as the coding blocklength goes to infinity. This work proposes a new paradigm, referred to as many-user information…
The interplay between nonlinear dynamic systems and noise has proved to be of great relevance in several application areas. In this presentation, we focus on the areas of information transmission and storage. We review some recent results…
The problem of state communication over a discrete memoryless channel with discrete memoryless state is studied when the state information is available strictly causally at the encoder. It is shown that block Markov encoding, in which the…
Fluctuations in biochemical networks, e.g., in a living cell, have a complex origin that precludes a description of such systems in terms of bipartite or multipartite processes, as is usually done in the framework of stochastic and/or…
We derive information-theoretic converses (i.e., lower bounds) for the minimum time required by any algorithm for distributed function computation over a network of point-to-point channels with finite capacity, where each node of the…
An additive noise channel is considered, in which the distribution of the noise is nonparametric and unknown. The problem of learning encoders and decoders based on noise samples is considered. For uncoded communication systems, the problem…
We formulate a model for intermittent communication that can capture bursty transmissions or a sporadically available channel, where in either case the receiver does not know a priori when the transmissions will occur. Focusing on the…
Shannon's information theory deliberately excludes message semantics. This paper develops a rigorous framework for semantic communication that integrates formal proof systems with Shannon-theoretic tools. We introduce an axiomatic…
Two familiar notions of correlation are rediscovered as the extreme operating points for distributed synthesis of a discrete memoryless channel, in which a stochastic channel output is generated based on a compressed description of the…