Related papers: Fisher Information and Mutual Information Constrai…
We prove that two well-known measures of information are interrelated in interesting and useful ways when applied to nonequilibrium circumstances. A nontrivial form of the lower bound for the Fisher information measure is derived in…
Famously, the quantum Fisher information -- the maximum Fisher information over all physical measurements -- is additive for independent copies of a system and the optimal measurement acts locally. We are left to wonder: does the same hold…
The Fisher information matrix summarizes the amount of information in a set of data relative to the quantities of interest. There are many applications of the information matrix in statistical modeling, system identification and parameter…
In many statistical applications that concern mathematical psychologists, the concept of Fisher information plays an important role. In this tutorial we clarify the concept of Fisher information as it manifests itself across three different…
Mixture-of-Experts models are commonly used when there exist distinct clusters with different relationships between the independent and dependent variables. Fitting such models for large datasets, however, is computationally virtually…
Many statistical models require an estimation of unknown (co)-variance parameter(s) in a model. The estimation usually obtained by maximizing a log-likelihood which involves log determinant terms. In principle, one requires the…
The Fisher information matrix (FIM) has long been of interest in statistics and other areas. It is widely used to measure the amount of information and calculate the lower bound for the variance for maximum likelihood estimation (MLE). In…
We study the problem of parameter estimation in time series stemming from general stochastic processes, where the outcomes may exhibit arbitrary temporal correlations. In particular, we address the question of how much Fisher information is…
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs…
The basic properties of the Fisher information allow to reveal the statistical meaning of classical inequalities between mean functions. The properties applied to scale mixtures of Gaussian distributions lead to a new mean function of…
Let $X$ be a non-negative random variable and let the conditional distribution of a random variable $Y$, given $X$, be ${Poisson}(\gamma \cdot X)$, for a parameter $\gamma \geq 0$. We identify a natural loss function such that: 1) The…
We present a unifying framework that bridges Bayesian asymptotics and information theory to analyze the asymptotic Shannon capacity of general large-scale MIMO channels including ones with nonlinearities or imperfect hardware. We derive…
We derive lower bounds on the Bayes risk in decentralized estimation, where the estimator does not have direct access to the random samples generated conditionally on the random parameter of interest, but only to the data received from…
We prove an extended convexity for quantum Fisher information of a mixed state with a given convex decomposition. This convexity introduces a bound which has two parts: i. classical part associated to the Fisher information of the…
Consider the centered Gaussian vector $X$ in $\R^n$ with covariance matrix $ \Sigma.$ Randomize $\Sigma$ such that $ \Sigma^{-1}$ has a Wishart distribution with shape parameter $p>(n-1)/2$ and mean $p\sigma.$ We compute the density…
We derive information-theoretic converses (i.e., lower bounds) for the minimum time required by any algorithm for distributed function computation over a network of point-to-point channels with finite capacity, where each node of the…
Estimating mutual information (MI) is a fundamental yet challenging task in data science and machine learning. This work proposes a new estimator for mutual information. Our main discovery is that a preliminary estimate of the data…
Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual…
Recently new approaches for sensing the frequency of time dependent Hamiltonians have been presented, and it was shown that the optimal Fisher information scales as $T^{4}.$ We present here our interpretation of this new scaling, where the…
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…