Related papers: Fisher Information and Mutual Information Constrai…
Fisher information is a measure of the best precision with which a parameter can be estimated from statistical data. It can also be defined for a continuous random variable without reference to any parameters, in which case it has a…
The Fisher information matrix can be used to characterize the local geometry of the parameter space of neural networks. It elucidates insightful theories and useful tools to understand and optimize neural networks. Given its high…
This paper studies semiparametric Fisher information in models parametrized by general normed spaces. The main contribution is to establish that positive semiparametric Fisher information is equivalent to the gradient of the parameter of…
Information theory is a powerful framework to capture aspects of dynamical systems with multiple degrees of freedom. Mathematically, the dynamics can be represented as a continuous curve $\mathcal{C}$ on a suitable hyperplane in flat space…
Fisher information, lies at the heart of parameter estimation theory, was recently found to have a close relation with multipartite entanglement (Pezz\'{e} and Smerzi, Phys. Rev. Lett. 102, 100401). We use Fisher information to distinguish…
Mutual information is fundamentally important for measuring statistical dependence between variables and for quantifying information transfer by signaling and communication mechanisms. It can, however, be challenging to evaluate for…
In the present paper, we would like to draw attention to a possible generalized Fisher information that fits well in the formalism of nonextensive thermostatistics. This generalized Fisher information is defined for densities on…
Fisher's criterion is a widely used tool in machine learning for feature selection. For large search spaces, Fisher's criterion can provide a scalable solution to select features. A challenging limitation of Fisher's criterion, however, is…
The most effective differentially private machine learning algorithms in practice rely on an additional source of purportedly public data. This paradigm is most interesting when the two sources combine to be more than the sum of their…
An important notion of common information between two random variables is due to Wyner. In this paper, we derive a lower bound on Wyner's common information for continuous random variables. The new bound improves on the only other general…
We prove two lower bounds for the complexity of non-log-concave sampling within the framework of Balasubramanian et al. (2022), who introduced the use of Fisher information (FI) bounds as a notion of approximate first-order stationarity in…
This paper studies the asymptotic behavior of the Fisher information for a Levy process discretely sampled at an increasing frequency. We show that it is possible to distinguish not only the continuous part of the process from its jumps…
We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections. A few examples where this problem is relevant are compressed sensing, sparse superposition codes, and code division multiple access.…
Given finite-dimensional random vectors $Y$, $X$, and $Z$ that form a Markov chain in that order (i.e., $Y \to X \to Z$), we derive upper bounds on the excess minimum risk using generalized information divergence measures. Here, $Y$ is a…
The Fisher Information matrix is a widely used measure for applications ranging from statistical inference, information geometry, experiment design, to the study of criticality in biological systems. Yet there is no commonly accepted…
Estimating mutual information accurately is pivotal across diverse applications, from machine learning to communications and biology, enabling us to gain insights into the inner mechanisms of complex systems. Yet, dealing with…
We derive an approximate expression for mutual information in a broad class of discrete-time stationary channels with continuous input, under the constraint of vanishing input amplitude or power. The approximation describes the input by its…
Uncertain input of a mathematical model induces uncertainties in the output and probabilistic sensitivity analysis identifies the influential inputs to guide decision-making. Of practical concern is the probability that the output would, or…
Point estimators may not exist, need not be unique, and their distributions are not parameter invariant. Generalized estimators provide distributions that are parameter invariant, unique, and exist when point estimates do not. Comparing…
Quantum Fisher information places the fundamental limit to the accuracy of estimating an unknown parameter. Here we shall provide the quantum Fisher information an operational meaning: a mixed state can be so prepared that a given…