Related papers: Some new insights into information decomposition i…
Reliably estimating the uncertainty of a prediction throughout the model lifecycle is crucial in many safety-critical applications. The most common way to measure this uncertainty is via the predicted confidence. While this tends to work…
Extracting useful information from data is a fundamental challenge across disciplines as diverse as climate, neuroscience, genetics, and ecology. In the era of ``big data'', data is ubiquitous, but appropriated methods are needed for…
Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables. However,…
We study the role of the information deposited in the environment of an open quantum system in course of the decoherence process. Redundant spreading of information -- the fact that some observables of the system can be independently…
We investigate the information distribution among different entities in the weak measurements protocol. Focusing on multilevel, decaying systems under continuous (no-click) monitoring, we derive exact, conservation-type information…
Notwithstanding various attempts to construct a Partial Information Decomposition (PID) for multiple variables by defining synergistic, redundant, and unique information, there is no consensus on how one ought to precisely define either of…
Mutual information $I(X;Y)$ is a useful definition in information theory to estimate how much information the random variable $Y$ holds about the random variable $X$. One way to define the mutual information is by comparing the joint…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
Constraint-based causal discovery from limited data is a notoriously difficult challenge due to the many borderline independence test decisions. Several approaches to improve the reliability of the predictions by exploiting redundancy in…
We address three outstanding problems in information theory. Problem one is the definition of a non-negative decomposition of the information conveyed by two or more sources about a target variable into the specific contribution of each…
We introduce a novel measure of dependence that captures the extent to which a random variable $Y$ is determined by a random vector $X$. The measure equals zero precisely when $Y$ and $X$ are independent, and it attains one exactly when $Y$…
We study the informational content of factor structures in discrete triangular systems. Factor structures have been employed in a variety of settings in cross sectional and panel data models, and in this paper we formally quantify their…
We consider lossy compression of an information source when the decoder has lossless access to a correlated one. This setup, also known as the Wyner-Ziv problem, is a special case of distributed source coding. To this day, real-world…
This paper considers an empirical likelihood inference for parameters defined by general estimating equations, when data are missing at random. The efficiency of existing estimators depends critically on correctly specifying the conditional…
Firstly, assuming Gaussianity, equations for the following information theory measures are presented: total correlation/coherence (TC), dual total correlation/coherence (DTC), O-information, TSE complexity, and redundancy-synergy index…
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual…
Causal inference studies whether the presence of a variable influences an observed outcome. As measured by quantities such as the "average treatment effect," this paradigm is employed across numerous biological fields, from vaccine and drug…
We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging…
Combining information from multiple samples is often needed in biomedical and economic studies, but the differences between these samples must be appropriately taken into account in the analysis of the combined data. We study estimation for…
The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model…