Related papers: The Relation Between Bayesian Fisher Information a…
The van Trees inequality relates the Ensemble Mean Squared Error of an estimator to a Bayesian version of the Fisher Information. The Ziv-Zakai inequality relates the Ensemble Mean Squared Error of an estimator to the Minimum Probability of…
In information theory, Fisher information and Shannon information (entropy) are respectively used to quantify the uncertainty associated with the distribution modeling and the uncertainty in specifying the outcome of given variables. These…
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
We consider the processing of statistical samples $X\sim P_\theta$ by a channel $p(y|x)$, and characterize how the statistical information from the samples for estimating the parameter $\theta\in\mathbb{R}^d$ can scale with the mutual…
I present several new relations between mutual information (MI) and statistical estimation error for a system that can be regarded simultaneously as a communication channel and as an estimator of an input parameter. I first derive a…
Markov Random Field models are powerful tools for the study of complex systems. However, little is known about how the interactions between the elements of such systems are encoded, especially from an information-theoretic perspective. In…
A relationship between the Fisher information and the characteristic function is established with the help of two inequalities. A necessary and sufficient condition for equality is found. These results are used to determine the asymptotic…
Fine-tuning and naturalness, the sensitivity of low-energy observables to small changes in the fundamental parameters of a theory, are cornerstones of physics beyond the Standard Model. We propose a new measure of fine-tuning based on…
Fisher information, Shannon information entropy and Statistical Complexity are calculated for the interface of a normal metal and a superconductor, as a function of the temperature for several materials. The order parameter $\Psi({\bf r})$…
The estimation of continuous parameters from measured data plays a central role in many fields of physics. A key tool in understanding and improving such estimation processes is the concept of Fisher information, which quantifies how…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
In this chapter, a statistical measure of complexity and the Fisher-Shannon information product are introduced and their properties are discussed. These measures are based on the interplay between the Shannon information, or a function of…
In this work the primary objective is to maximize the precision of the maximum likelihood estimate in a linear regression model through the efficient design of the experiment. One common measure of precision is the unconditional mean square…
This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. First, an estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. Second, a new…
We highlight the advantages of using simultaneously the Shannon and Fisher information measures in providing a useful form of the uncertainty relation for the position-momentum case. It does not require any Fourier transformation. The…
Informationally complete measurements form the foundation of universal quantum state reconstruction, while quantum parameter estimation is based on the local structure of the manifold of quantum states. Here we establish a general link…
Shannon entropy is the most crucial foundation of Information Theory, which has been proven to be effective in many fields such as communications. Renyi entropy and Chernoff information are other two popular measures of information with…
Fisher information is a measure of the best precision with which a parameter can be estimated from statistical data. It can also be defined for a continuous random variable without reference to any parameters, in which case it has a…
Structured optical beams possess rich spatial features that are commonly characterized using entropic measures of field complexity. However, such measures do not directly quantify the operational usefulness of optical structure for…
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…