Related papers: A Nonstochastic Information Theory for Communicati…
Using the concept of discrete noiseless channels, it was shown by Shannon in A Mathematical Theory of Communication that the ultimate performance of an encoder for a constrained system is limited by the combinatorial capacity of the system…
There is a fundamental limit to what is knowable about atomic and molecular scale systems. This fuzziness is not always due to the act of measurement. Other contributing factors include system parameter uncertainty, functional uncertainty…
The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are…
The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions $\big\{{\bf P}_{B_i|B^{i-1}, A_i} :i=0, 1, \ldots, n\big\}$ and $\big\{ {\bf P}_{B_i|B_{i-M}^{i-1}, A_i} :i=0,…
A central concept in the connection between physics and information theory is entropy, which represents the amount of information extracted from the system by the observer performing measurements in an experiment. Indeed, Jaynes' principle…
A fundamental concept in control theory is that of controllability, where any system state can be reached through an appropriate choice of control inputs. Indeed, a large body of classical and modern approaches are designed for controllable…
Opacity, or non-interference, is a property ensuring that an external observer cannot infer confidential information (the "secret") from system observations. We introduce an information-theoretic measure of opacity, which quantifies…
The question of deriving general force/flux relationships that apply out of the linear response regime is a central topic of theories for nonequilibrium statistical mechanics. This work applies an information theory perspective to compute…
Although compartmental dynamical systems are used in many different areas of science, model selection based on the maximum entropy principle (MaxEnt) is challenging because of the lack of methods for quantifying the entropy for this type of…
We reconsider the persistence of information under the dynamics of the logistic map in order to discuss communication through a nonlinear channel where the sender can set the initial state of the system with finite resolution, and the…
The increasing amount of controllable generation and consumption in distribution grids poses a severe challenge in keeping voltage values within admissible ranges. Existing approaches have considered different optimal power flow…
Despite being optimized, the information processing of biological organisms exhibits significant variability in its complexity and capability. One potential source of this diversity is the limitation of resources required for information…
We propose a general framework, dubbed Stochastic Processing under Imperfect Information (SPII), to study the impact of information constraints and memories on dynamic resource allocation. The framework involves a Stochastic Processing…
One of the crucial steps in scientific studies is to specify dependent relationships among factors in a system of interest. Given little knowledge of a system, can we characterize the underlying dependent relationships through observation…
In this paper, a network-based stochastic information propagation model is developed. The information flow is modeled by a probabilistic differential equation system. The numerical solution of these equations leads to the expected number of…
We consider the problem of communication over a network containing a hidden and malicious adversary that can control a subset of network resources, and aims to disrupt communications. We focus on omniscient node-based adversaries, i.e., the…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
We derive an approximate expression for mutual information in a broad class of discrete-time stationary channels with continuous input, under the constraint of vanishing input amplitude or power. The approximation describes the input by its…
In this article, we establish a comprehensive theoretical framework for remote estimation in a networked system composed of a source that is observed by a sensor, a remote monitor that needs to estimate the state of the source in real time,…
We model the spread of information in a homogeneously mixed population using the Maki Thompson rumor model. We formulate an optimal control problem, from the perspective of single campaigner, to maximize the spread of information when the…