Related papers: Fisher Information and Mutual Information Constrai…
We derive a general upper bound to mutual information in terms of the Fisher information. The bound may be further used to derive a lower bound for the Bayesian quadratic cost. These two provide alternatives to other inequalities in the…
We derive general upper bounds to pointwise mutual information in terms of stochastic Fisher information and show these bounds average to known results in the literature for bounds to mutual information in terms of Fisher information. These…
We prove lower bounds on the error of any estimator for the mean of a real probability distribution under the knowledge that the distribution belongs to a given set. We apply these lower bounds both to parametric and nonparametric…
We consider the problem of learning high-dimensional, nonparametric and structured (e.g. Gaussian) distributions in distributed networks, where each node in the network observes an independent sample from the underlying distribution and can…
The mean of an unknown variance-$\sigma^2$ distribution $f$ can be estimated from $n$ samples with variance $\frac{\sigma^2}{n}$ and nearly corresponding subgaussian rate. When $f$ is known up to translation, this can be improved…
This paper deals with the problem of estimating the coupling constant $\theta$ of a mixing quantum Markov chain. For a repeated measurement on the chain's output we show that the outcomes' time average has an asymptotically normal…
The Fisher information matrix (FIM) plays an important role in the analysis of parameter inference and system design problems. In a number of cases, however, the statistical data distribution and its associated information matrix are either…
A relationship between the Fisher information and the characteristic function is established with the help of two inequalities. A necessary and sufficient condition for equality is found. These results are used to determine the asymptotic…
We present a numerical method to evaluate mutual information (MI) in nonlinear Gaussian noise channels by using denoising score matching (DSM) learning for estimating the score function of channel output. Via de Bruijn's identity, Fisher…
The problem of determining the intrinsic quality of a signal processing system with respect to the inference of an unknown deterministic parameter $\theta$ is considered. While the Fisher information measure $F(\theta)$ forms a classical…
Within the framework of linear vector Gaussian channels with arbitrary signaling, closed-form expressions for the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system…
The Fisher information matrix is a quantity of fundamental importance for information geometry and asymptotic statistics. In practice, it is widely used to quickly estimate the expected information available in a data set and guide…
I present several new relations between mutual information (MI) and statistical estimation error for a system that can be regarded simultaneously as a communication channel and as an estimator of an input parameter. I first derive a…
We consider a distributed logistic regression problem where labeled data pairs $(X_i,Y_i)\in \mathbb{R}^d\times\{-1,1\}$ for $i=1,\ldots,n$ are distributed across multiple machines in a network and must be communicated to a centralized…
We develop data processing inequalities that describe how Fisher information from statistical samples can scale with the privacy parameter $\varepsilon$ under local differential privacy constraints. These bounds are valid under general…
Motivated by the information bound for the asymptotic variance of M-estimates for scale, we define Fisher information of scale of any distribution function F on the real line as a suitable supremum. In addition, we enforce equivariance by a…
We establish an expansion by Gamma-convergence of the Fisher information relative to the reference measure exp(-beta V), where V is a generic multiwell potential and beta goes to infinity. The expansion reveals a hierarchy of multiple…
In this paper, we first establish general bounds on the Fisher information distance to the class of normal distributions of Malliavin differentiable random variables. We then study the rate of Fisher information convergence in the central…
We study the problem of estimating the magnitude of a Gaussian beam displacement using a two pixel or 'split' detector. We calculate the maximum likelihood estimator, and compute its asymptotic mean-squared-error via the Fisher information.…
This paper considers the model of an arbitrary distributed signal x observed through an added independent white Gaussian noise w, y=x+w. New relations between the minimal mean square error of the non-causal estimator and the likelihood…