Related papers: Pointwise mutual information bounded by stochastic…
Fisher's criterion is a widely used tool in machine learning for feature selection. For large search spaces, Fisher's criterion can provide a scalable solution to select features. A challenging limitation of Fisher's criterion, however, is…
Some of the tightest information-theoretic generalization bounds depend on the average information between the learned hypothesis and a single training example. However, these sample-wise bounds were derived only for expected generalization…
This note reviews a recent contribution about the Fisher information for the space-homogeneous Boltzmann equation by L. Silvestre, C. Villani and the author (arXiv, 2024). This classical functional from information theory is shown to be…
We propose a modified $\chi^{\beta}$-divergence, give some of its properties, and show that this leads to the definition of a generalized Fisher information. We give generalized Cram\'er-Rao inequalities, involving this Fisher information,…
We present a novel and simple method to numerically calculate Fisher Information Matrices for stochastic chemical kinetics models. The linear noise approximation is used to derive model equations and a likelihood function which leads to an…
A fundamental question in theoretical machine learning is generalization. Over the past decades, the PAC-Bayesian approach has been established as a flexible framework to address the generalization capabilities of machine learning…
We derive explicit expressions for the quantum Fisher information and the symmetric logarithmic derivative (SLD) of a quantum state in the exponential form; the SLD is expressed in terms of the generator. Applications include…
The quantum Cram\'er-Rao bound sets a fundamental limit on the accuracy of unbiased parameter estimation in quantum systems, relating the uncertainty in determining a parameter to the inverse of the quantum Fisher information. We…
The convergence to stable laws is studied in relative Fisher information for sums of i.i.d. random variables.
The Fisher information matrix summarizes the amount of information in a set of data relative to the quantities of interest. There are many applications of the information matrix in statistical modeling, system identification and parameter…
We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm. These bounds improve over the existing…
In this paper, we investigate the stochastic thermodynamics of Fisher information (FI), meaning we characterize both the \textit{fluctuations} of FI, introducing a parastatistics of that quantity, and thermodynamic quantities. We introduce…
This paper studies the asymptotic behavior of the Fisher information for a Levy process discretely sampled at an increasing frequency. We show that it is possible to distinguish not only the continuous part of the process from its jumps…
Simultaneous estimation of multiple parameters is required in many practical applications. A lower bound on the variance of simultaneous estimation is given by the quantum Fisher information matrix. This lower bound is, however, not…
The correlation distance quantifies the statistical independence of two classical or quantum systems, via the distance from their joint state to the product of the marginal states. Tight lower bounds are given for the mutual information…
The accessible information quantifies the amount of classical information that can be extracted from an ensemble of quantum states. Analogously, the informational power quantifies the amount of classical information that can be extracted by…
Fisher Information is a key notion in the whole field of quantum metrology. It allows for a direct quantification of maximal achievable precision of estimation of parameters encoded in quantum states using the most general quantum…
While a positive operator valued measure gives the probabilities in a quantum measurement, an instrument gives both the probabilities and the a posteriori states. By interpreting the instrument as a quantum channel and by using the…
This paper deals with Cram\'er-Rao inequalities in the context of nonextensive statistics and in estimation theory. It gives characterizations of generalized q-Gaussian distributions, and introduces generalized versions of Fisher…
We consider kinetic models for Fermi-Dirac-like particles obeying the exclusion principle. A generalized notion of Fisher information, tailored to kinetic equations of Fermi-Dirac-Fokker-Planck type, is introduced via the associated entropy…