Related papers: Chentsov's theorem for exponential families
We establish a previously unexplored conservation law for the Quantum Fisher Information Matrix (QFIM) expressed as follows; when the QFIM is constructed from a set of observables closed under commutation, i.e., a Lie algebra, the spectrum…
We consider the central limit theorem for stable laws in the case of the standardized sum of independent and identically distributed random variables with regular probability density function. By showing decay of different entropy…
Quantum Fisher information matrices (QFIMs) are fundamental to estimation theory: they encode the ultimate limit for the sensitivity with which a set of parameters can be estimated using a given probe. Since the limit invokes the inverse of…
Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their $q$-deformation; when $q$ is a prime power, these $q$-multinomial coefficients…
The Ising model is a celebrated example of a Markov random field, introduced in statistical physics to model ferromagnetism. This is a discrete exponential family with binary outcomes, where the sufficient statistic involves a quadratic…
Fisher information is a lower bound on the uncertainty in the statistical estimation of classical and quantum mechanical parameters. While some deterministic dynamical systems are not subject to random fluctuations, they do still have a…
The Chernoff bound is one of the most widely used tools in theoretical computer science. It's rare to find a randomized algorithm that doesn't employ a Chernoff bound in its analysis. The standard proofs of Chernoff bounds are beautiful but…
A unified presentation of the perturbation and variational methods for the generalized statistical mechanics based on Tsallis entropy is given here. In the case of the variational method, the Bogoliubov inequality is generalized in a very…
A new information-theoretic approach to the central limit theorem for stable laws is presented. The main novelty is the concept of relative fractional Fisher information, which shares most of the properties of the classical one, included…
Exponential families encompass the distributions central to modern machine learning -- softmax, Gaussians, and Boltzmann distributions -- and underlie the theory of variational inference, entropy-regularized reinforcement learning, and…
The Fisher information metric or the Fisher-Rao metric corresponds to a natural Riemannian metric defined on a parameterized family of probability density functions. As in the case of Riemannian geometry, we can define a distance in terms…
Chern-Simons gauge theory, since its inception as a topological quantum field theory, has proved to be a rich source of understanding for knot invariants. In this work the theory is used to explore the definition of the expectation value of…
Statistical inference may follow a frequentist approach or it may follow a Bayesian approach or it may use the minimum description length principle (MDL). Our goal is to identify situations in which these different approaches to statistical…
Quantum Fisher information places the fundamental limit to the accuracy of estimating an unknown parameter. Here we shall provide the quantum Fisher information an operational meaning: a mixed state can be so prepared that a given…
Hallin and Ley [Bernoulli 18 (2012) 747-763] investigate and fully characterize the Fisher singularity phenomenon in univariate and multivariate families of skew-symmetric distributions. This paper proposes a refined analysis of the…
The purpose of this note is to recall one remarkable theorem of Khinchin about the special role of the Gaussian distribution. This theorem allows us to give a new interpretation of the Lindeberg condition: it guarantees the uniform…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the…
When dealing with a parametric statistical model, a Riemannian manifold can naturally appear by endowing the parameter space with the Fisher information metric. The geometry induced on the parameters by this metric is then referred to as…
We show that external randomization may enforce the convergence of test statistics to their limiting distributions in particular cases. This results in a sharper inference. Our approach is based on a central limit theorem for weighted sums.…
Entropy is a key measure in studies related to information theory and its many applications. Campbell of the first time recognized that exponential of Shannons entropy is just the size of the sample space when the distribution is uniform.…