Related papers: Fisher Information and Mutual Information Constrai…
We consider the problems of clustering, classification, and visualization of high-dimensional data when no straightforward Euclidean representation exists. Typically, these tasks are performed by first reducing the high-dimensional data to…
We propose a method for estimating the Fisher score--the gradient of the log-likelihood with respect to model parameters--using score matching. By introducing a latent parameter model, we show that the Fisher score can be learned by…
We study data processing inequalities that are derived from a certain class of generalized information measures, where a series of convex functions and multiplicative likelihood ratios are nested alternately. While these information…
We consider a high-dimensional mean estimation problem over a binary hidden Markov model, which illuminates the interplay between memory in data, sample size, dimension, and signal strength in statistical inference. In this model, an…
Fundamental relations between information and estimation have been established in the literature for the discrete-time Gaussian and Poisson channels. In this work, we demonstrate that such relations hold for a much larger class of…
Postselected weak measurement has shown significant potential for detecting small physical effects due to its unique weak-value-amplification phenomenon. Previous works suggest that Heisenberg-limit precision can be attained using only the…
Natural gradient descent, which preconditions a gradient descent update with the Fisher information matrix of the underlying statistical model, is a way to capture partial second-order information. Several highly visible works have…
We prove lower bounds on the number of samples needed to privately estimate the covariance matrix of a Gaussian distribution. Our bounds match existing upper bounds in the widest known setting of parameters. Our analysis relies on the…
The Fisher information matrix (FIM) is a key quantity in statistics as it is required for example for evaluating asymptotic precisions of parameter estimates, for computing test statistics or asymptotic distributions in statistical testing,…
In this brief paper we revisit the Fisher information content of cosmological power spectra or two-point functions of Gaussian fields in order to comment on the assumption of Gaussian estimators and the use of parameter-dependent covariance…
Estimating mutual information (MI) from samples is a fundamental problem in statistics, machine learning, and data analysis. Recently it was shown that a popular class of non-parametric MI estimators perform very poorly for strongly…
In multiparameter quantum metrology, the weighted-arithmetic-mean error of estimation is often used as a scalar cost function to be minimized during design optimization. However, other types of mean error can reveal different facets of…
This paper builds upon the methods developed in [22] and [15] to investigate the large population behavior of non exchangeable systems of N diffusive particles when the interaction matrix converges (in some sense) to a graphon. We first…
Hierarchical parametric models consisting of observable and latent variables are widely used for unsupervised learning tasks. For example, a mixture model is a representative hierarchical model for clustering. From the statistical point of…
We establish an a priori estimate for the dissipation of the Fisher information for the space-homogeneous Landau equation with very soft potentials. This work is motivated by the recent breakthrough by Guillen and Silvestre, which proves…
The importance of Fisher information is increasing in nonequilibrium thermodynamics, as it has played a fundamental role in trade-off relations such as thermodynamic uncertainty relations and speed limits. In this work, we investigate…
Exponential models of distributions are widely used in machine learning for classiffication and modelling. It is well known that they can be interpreted as maximum entropy models under empirical expectation constraints. In this work, we…
We show that the mathematical form of the information measure of Fisher's I for a Gibbs' canonical probability distribution (the most important one in statistical mechanics) incorporates important features of the intrinsic structure of…
We study the problem of community detection when there is covariate information about the node labels and one observes multiple correlated networks. We provide an asymptotic upper bound on the per-node mutual information as well as a…
The unavoidable interaction between a quantum system and the external noisy environment can be mimicked by a sequence of stochastic measurements whose outcomes are neglected. Here we investigate how this stochasticity is reflected in the…