Related papers: Information-optimal mixing at low Reynolds number
Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution with minimal redundancy. Finding an optimal measurement is…
Information processing typically occurs via the composition of modular units, such as universal logic gates. The benefit of modular information processing, in contrast to globally integrated information processing, is that complex global…
We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation, which is effective to identify the feature-level algorithmic bias by taking advantage of conditional mutual information.…
We rigorously derive a single-letter variational expression for the mutual information of the asymmetric two-groups stochastic block model in the dense graph regime. Existing proofs in the literature are indirect, as they involve mapping…
How quantum information is scrambled in the global degrees of freedom of non-equilibrium many-body systems is a key question to understand local thermalization. Here we propose that the scaling of the mutual information between two…
We study the compressive diffusion strategies over distributed networks based on the diffusion implementation and adaptive extraction of the information from the compressed diffusion data. We demonstrate that one can achieve a comparable…
We propose a flexible gradient tracking approach with adjustable computation and communication steps for solving distributed stochastic optimization problem over networks. The proposed method allows each node to perform multiple local…
We consider two-dimensional homogeneous shear turbulence within the context of optimal control, a multi-scale turbulence model containing the fluctuation velocity and pressure correlations up to the fourth order; The model is formulated on…
Mutual information (MI) is a fundamental quantity in information theory and machine learning. However, direct estimation of MI is intractable, even if the true joint probability density for the variables of interest is known, as it involves…
Biological organisms are open, adaptve systems that can respond to changes in environment in specific ways. Adaptation and response can be posed as an optimization problem, with a tradeoff between the benefit obtained from a response and…
In the context of distributed estimation, we consider the problem of sensor collaboration, which refers to the act of sharing measurements with neighboring sensors prior to transmission to a fusion center. While incorporating the cost of…
The ability of machine learning (ML) algorithms to generalize well to unseen data has been studied through the lens of information theory, by bounding the generalization error with the input-output mutual information (MI), i.e., the MI…
Let $P_Z$ be a given distribution on $\mathbb{R}^n$. For any $y\in\mathbb{R}^n$, we may interpret $\rho(y):=\ln\mathbb{E}[e^{\left<y,Z\right>}]$ as a soft-max of $\left<y,Z\right>$. We explore lower bounds on $\mathbb{E}[\rho(Y)]$ in terms…
We develop numerical and analytical approaches to calculate mutual information between complete paths of two molecular components embedded into a larger reaction network. In particular, we focus on a continuous-time Markov chain formalism,…
Stochastic optimisation algorithms are the de facto standard for machine learning with large amounts of data. Handling only a subset of available data in each optimisation step dramatically reduces the per-iteration computational costs,…
We present a closed-form expression for the minimal delay that is achievable in a setting that combines a buffer and an erasure code, used to mitigate the packet delay variance. The erasure code is modeled according to the recent…
Consider a joint quantum state of a system and its environment. A measurement on the environment induces a decomposition of the system state. Using algorithmic information theory, we define the preparation information of a pure or mixed…
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…
A recent first-principles approach to the non-linear rheology of dense colloidal suspensions is evaluated and compared to simulation results of sheared systems close to their glass transitions. The predicted scenario of a universal…
In network science, researchers often use mutual information to understand the difference between network partitions produced by community detection methods. Here we extend the use of mutual information to covers, that is, the cases where a…