Related papers: On mutual information estimation for mixed-pair ra…
Consider statistical learning (e.g. discrete distribution estimation) with local $\epsilon$-differential privacy, which preserves each data provider's privacy locally, we aim to optimize statistical data utility under the privacy…
In this paper we build a joint model which can accommodate for binary, ordinal and continuous responses, by assuming that the errors of the continuous variables and the errors underlying the ordinal and binary outcomes follow a multivariate…
This paper develops some objective priors for certain parameters of the bivariate normal distribution. The parameters considered are the regression coefficient, the generalized variance, and the ratio of the conditional variance of one…
In this paper, we propose an eigenvalue analysis -- of system dynamics models -- based on the Mutual Information measure, which in turn will be estimated via the Kernel Density Estimation method. We postulate that the proposed approach…
We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access.…
The conditional mutual information quantifies the conditional dependence of two random variables. It has numerous applications; it forms, for example, part of the definition of transfer entropy, a common measure of the causal relationship…
Bimodal truncated count distributions are frequently observed in aggregate survey data and in user ratings when respondents are mixed in their opinion. They also arise in censored count data, where the highest category might create an…
We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning models. To accurately estimate MI from a finite number of samples, we present GMM-MI…
The integrated information theory is thought to be a key clue towards the theoretical understanding of consciousness. In this study, we propose a simple numerical model comprising a set of coupled double quantum dots, where the…
Propensity score matching is commonly used to draw causal inference from observational survival data. However, its asymptotic properties have yet to be established, and variance estimation is still open to debate. We derive the statistical…
An extension of the empirical copula is considered by combining an estimator of a multivariate cumulative distribution function with estimators of the marginal cumulative distribution functions for marginal estimators that are not…
We characterize mutual information as the unique map on ordered pairs of random variables satisfying a set of axioms similar to those of Faddeev's characterization of the Shannon entropy. There is a new axiom in our characterization however…
For discrete random variables X_1,..., X_n we construct an n by n matrix. In the (i,j) entry we put the mutual information I(X_i;X_j) between X_i and X_j. In particular, in the (i,i) entry we put the entropy H(X_i)=I(X_i;X_i) of X_i. This…
In some estimation problems, especially in applications dealing with information theory, signal processing and biology, theory provides us with additional information allowing us to restrict the parameter space to a finite number of points.…
Our contribution is to widen the scope of extreme value analysis applied to discrete-valued data. Extreme values of a random variable $X$ are commonly modeled using the generalized Pareto distribution, a method that often gives good results…
We study the generic scaling properties of the mutual information between two disjoint intervals, in a class of one-dimensional quantum critical systems described by the c=1 bosonic field theory. A numerical analysis of a spin-chain model…
We propose a test of independence of two multivariate random vectors, given a sample from the underlying population. Our approach, which we call MINT, is based on the estimation of mutual information, whose decomposition into joint and…
Mutual information is a measure of the dependence between random variables that has been used successfully in myriad applications in many fields. Generalized mutual information measures that go beyond classical Shannon mutual information…
Neural networks and other machine learning models compute continuous representations, while humans communicate mostly through discrete symbols. Reconciling these two forms of communication is desirable for generating human-readable…
In this paper we study the problem of statistical inference on the parameters of the semiparametric variance-mean mixtures. This class of mixtures has recently become rather popular in statistical and financial modelling. We design a…