Related papers: On the Information of the Second Moments Between R…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
We propose a unifying phase-space approach to the construction of mutually unbiased bases for a two-qubit system. It is based on an explicit classification of the geometrical structures compatible with the notion of unbiasedness. These…
In many data analyses, each measurement may come with a simple yes/no correction; for example, belonging to one of two populations or being contaminated or not. Ignoring such binary effects may bias the results, while accounting for them…
By combining a bound on the absolute value of the difference of mutual information between two joint probablity distributions with a fixed variational distance, and a bound on the probability of a maximal deviation in variational distance…
In the analysis of time series from nonlinear sources, mutual information (MI) is used as a nonlinear statistical criterion for the selection of an appropriate time delay in time delay reconstruction of the state space. MI is a statistic…
Bayesian model comparison (BMC) offers a principled probabilistic approach to study and rank competing models. In standard BMC, we construct a discrete probability distribution over the set of possible models, conditional on the observed…
We derive a well-defined renormalized version of mutual information that allows to estimate the dependence between continuous random variables in the important case when one is deterministically dependent on the other. This is the situation…
An important challenge in statistical analysis concerns the control of the finite sample bias of estimators. For example, the maximum likelihood estimator has a bias that can result in a significant inferential loss. This problem is…
The framework of Partial Information Decomposition (PID) unveils complex nonlinear interactions in network systems by dissecting the mutual information (MI) between a target variable and several source variables. While PID measures have…
Graphical models have proven to be powerful tools for representing high-dimensional systems of random variables. One example of such a model is the undirected graph, in which lack of an edge represents conditional independence between two…
Estimating mutual information from observed samples is a basic primitive, useful in several machine learning tasks including correlation mining, information bottleneck clustering, learning a Chow-Liu tree, and conditional independence…
When an optimal measurement is made on a qubit and what we call an Unbiased Mixture of the resulting ensembles is taken, then the post measurement density matrix is shown to be related to the pre-measurement density matrix through a simple…
Random factor graphs provide a powerful framework for the study of inference problems such as decoding problems or the stochastic block model. Information-theoretically the key quantity of interest is the mutual information between the…
This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased…
We consider the notion of unitary transformations forming bases for subspaces of $M(d,\mathbb{C})$ such that the square of Hilbert-Schmidt inner product of matrices from the differing bases is a constant. Moving from the qubit case,…
This thesis uses a quantity that is defined and justified by information theory -- mutual information -- to examine models of condensed matter systems. More precisely, it studies models which are made up out of ferromagnetically interacting…
Uncertainty quantification is a critical aspect of machine learning models, providing important insights into the reliability of predictions and aiding the decision-making process in real-world applications. This paper proposes a novel way…
A Bayesian method of moments/instrumental variable (BMOM/IV) approach is developed and applied in the analysis of the important mean and multiple regression models. Given a single set of data, it is shown how to obtain posterior and…
In this paper we develop a new technique to model joint distributions of signals. Our technique is based on quantum mechanical conjugate variables. We show that the transition probability of quantum states leads to a distance function on…
Estimating mutual information (MI) is a fundamental yet challenging task in data science and machine learning. This work proposes a new estimator for mutual information. Our main discovery is that a preliminary estimate of the data…