Related papers: Distribution of Mutual Information
Mechanistic Interpretability (MI) aims to reverse-engineer model behaviors by identifying functional sub-networks. Yet, the scientific validity of these findings depends on their stability. In this work, we argue that circuit discovery is…
Shannon defined the mutual information between two variables. We illustrate why the true mutual information between a variable and the predictions made by a prediction algorithm is not a suitable measure of prediction quality, but the…
Periodic patterns in dynamical behaviours of biological models described by simple form differential delay equations are studied. Mathematical models are given by a class of scalar delay differential equations with a multiplicative time…
Analysis of a probabilistic system often requires to learn the joint probability distribution of its random variables. The computation of the exact distribution is usually an exhaustive precise analysis on all executions of the system. To…
This paper investigates the Age of Incorrect Information (AoII) in a communication system whose channel suffers a random delay. We consider a slotted-time system where a transmitter observes a dynamic source and decides when to send updates…
There are numerous methods for detecting anomalies in time series, but that is only the first step to understanding them. We strive to exceed this by explaining those anomalies. Thus we develop a novel attribution scheme for multivariate…
Distributed averaging is among the most relevant cooperative control problems, with applications in sensor and robotic networks, distributed signal processing, data fusion, and load balancing. Consensus and gossip algorithms have been…
We consider the analysis of continuous repeated measurement outcomes that are collected through time, also known as longitudinal data. A standard framework for analysing data of this kind is a linear Gaussian mixed-effects model within…
Inverse problems are prevalent across various disciplines in science and engineering. In the field of computer vision, tasks such as inpainting, deblurring, and super-resolution are commonly formulated as inverse problems. Recently,…
Biochemistry, ecology, and neuroscience are examples of prominent fields aiming at describing interacting systems that exhibit non-trivial couplings to complex, ever-changing environments. We have recently shown that linear interactions and…
This paper considers the stationary distribution of the age of information (AoI) in information update systems. We first derive a general formula for the stationary distribution of the AoI, which holds for a wide class of information update…
A key metric to express the timeliness of status updates in latency-sensitive networked systems is the age of information (AoI), i.e., the time elapsed since the generation of the last received informative status message. This metric allows…
We consider the scattering of an electron from a semi-infinite one-dimensional random medium. The random medium is characterized by force, $-\d V/\d L$ being the basic random variable. We obtain an analytical expression for the stationary…
Estimating conditional mutual information (CMI) is an essential yet challenging step in many machine learning and data mining tasks. Estimating CMI from data that contains both discrete and continuous variables, or even discrete-continuous…
In this paper, time delay estimation techniques robust to narrowband interference (NBI) are proposed. Owing to the deluge of wireless signal interference these days, narrowband interference is a common problem for communication and…
We consider distributed average consensus in a wireless network with partial communication to reduce the number of transmissions in every iteration/round. Considering the broadcast nature of wireless channels, we propose a probabilistic…
In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the…
Information-theoretic arguments focus on modeling the reliability of information transmission, assuming availability of infinite data at sources, thus ignoring randomness in message generation times at the respective sources. However, in…
Within a random matrix theory approach, we obtain spectral statistics of the Wigner time delay matrix $Q$, for arbitrary channels number $M$ and for all symmetry classes, in fact for general Dyson parameter $\beta$. We also put forth two…
In this letter, we investigate the mutual information rate (MIR) achieved by an independent identically distributed (IID) Gaussian input on the intensity-driven signal transduction channel. Specifically, the asymptotic expression of the…