Related papers: Multivariate information measures: an experimental…
Information integration plays a pivotal role in biomedical studies by facilitating the combination and analysis of independent datasets from multiple studies, thereby uncovering valuable insights that might otherwise remain obscured due to…
We consider probabilistic theories in which the most elementary system, a two-dimensional system, contains one bit of information. The bit is assumed to be contained in any complete set of mutually complementary measurements. The…
Reliable estimation of predictive uncertainty is crucial for machine learning applications, particularly in high-stakes scenarios where hedging against risks is essential. Despite its significance, there is no universal agreement on how to…
The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are…
Many information sources are considered into data fusion in order to improve the decision in terms of uncertainty and imprecision. For each technique used for data fusion, the asumption on independance is usually made. We propose in this…
We carry out a systematic study of uncertainty measures that are generic to dynamical processes of varied origins, provided they induce suitable continuous probability distributions. The major technical tool are the information theory…
The integrated information theory of consciousness (IIT) is uniquely ambitious in proposing a mathematical formula, derived from apparently fundamental properties of conscious experience, to describe the quantity and quality of…
The question of selecting the "best" amongst different choices is a common problem in statistics. In drug development, our motivating setting, the question becomes, for example: what is the dose that gives me a pre-specified risk of…
Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that…
Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good…
One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of…
Model selection in clustering requires (i) to specify a suitable clustering principle and (ii) to control the model order complexity by choosing an appropriate number of clusters depending on the noise level in the data. We advocate an…
The purpose of this paper is to point out a new connection between information theory and dynamical systems. In the information theory side, we consider rate distortion theory, which studies lossy data compression of stochastic processes…
The traditional approach to the quantitative study of segregation is to employ indices that are selected by ``desirable properties''. Here, we detail how information theory underpins entropy-based indices and demonstrate how desirable…
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual information between the past and the future of a time series,…
The Integrated Information Theory (IIT) might be our current best bet at a scientific explanation of phenomenal consciousness. IIT focuses on the distinctively subjective and phenomenological aspects of conscious experience. Currently, it…
Multivariate datasets are common in various real-world applications. Recently, copulas have received significant attention for modeling dependencies among random variables. A copula-based information measure is required to quantify the…
Mutual information is a general statistical dependency measure which has found applications in representation learning, causality, domain generalization and computational biology. However, mutual information estimators are typically…
Methods for addressing missing data have become much more accessible to applied researchers. However, little guidance exists to help researchers systematically identify plausible missing data mechanisms in order to ensure that these methods…
For biological experiments aiming at calibrating models with unknown parameters, a good experimental design is crucial, especially for those subject to various constraints, such as financial limitations, time consumption and physical…