Related papers: Fisher information as a performance metric for loc…
Choosing a suitable loss function is essential when learning by empirical risk minimisation. In many practical cases, the datasets used for training a classifier may contain incorrect labels, which prompts the interest for using loss…
Obtaining labels can be costly and time-consuming. Active learning allows a learning algorithm to intelligently query samples to be labeled for efficient learning. Fisher information ratio (FIR) has been used as an objective for selecting…
The problem of determining the best achievable performance of arbitrary lossless compression algorithms is examined, when correlated side information is available at both the encoder and decoder. For arbitrary source-side information pairs,…
This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher…
We prove lower bounds on the error of any estimator for the mean of a real probability distribution under the knowledge that the distribution belongs to a given set. We apply these lower bounds both to parametric and nonparametric…
In this paper, we study an asymptotic approximation of the Fisher information for the estimation of a scalar parameter using quantized measurements. We show that, as the number of quantization intervals tends to infinity, the loss of Fisher…
We study limits for the detection and estimation of weak sinusoidal signals in the primary part of the mammalian auditory system using a stochastic Fitzhugh-Nagumo (FHN) model and an action-reaction model for synaptic plasticity. Our…
The technical merits of weak value amplification techniques are analyzed. We consider models of several different types of technical noise in an optical context and show that weak value amplification techniques (which only use a small…
A common assumption in signal processing is that underlying data numerically conforms to a Gaussian distribution. It is commonly utilized in signal processing to describe unknown additive noise in a system and is often justified by citing…
The Fisher information matrix (FIM) is a foundational concept in statistical signal processing. The FIM depends on the probability distribution, assumed to belong to a smooth parametric family. Traditional approaches to estimating the FIM…
This work proposes a new loss function targeting classification problems, utilizing a source of information overlooked by cross entropy loss. First, we derive a series of the tightest upper and lower bounds for the probability of a random…
Maximum likelihood estimates and corresponding confidence regions of the estimates are commonly used in statistical inference. In practice, people often construct approximate confidence regions with the Fisher information at given sample…
Mimicking the maximum likelihood estimator, we construct first order Cramer-Rao efficient and explicitly computable estimators for the scale parameter $\sigma^2$ in the model $Z_{i,n}=\sigma n^{-\beta}X_i+Y_i,i=1,\ldots,n,\beta>0$ with…
We study the optimality conditions of information transfer in systems with memory in the low signal-to-noise ratio regime of vanishing input amplitude. We find that the optimal mutual information is represented by a maximum-variance of the…
We introduce a positive Hermitian operator, the Fisher operator, and use it to examine a measurement process incorporating unitary dynamics and complete measurements. We develop the idea of information complement, the minimization of which…
The Fisher information matrix is a quantity of fundamental importance for information geometry and asymptotic statistics. In practice, it is widely used to quickly estimate the expected information available in a data set and guide…
This paper proposes feasible asymptotically efficient estimators for a certain class of Gaussian noises with self-similar and stationary properties, which includes the fractional Gaussian noise, under high frequency observations. In this…
For a given quantum state used in sensing, the quantum Cram\'er-Rao bound (QCRB) sets a fundamental limit on the precision achievable by an unbiased estimator of an unknown parameter, determined by the inverse of the quantum Fisher…
An approach based on the Fisher information (FI) is developed to quantify the maximum information gain and optimal experimental design in neutron reflectometry experiments. In these experiments, the FI can be analytically calculated and…
We consider the task of estimating a low-rank matrix from non-linear and noisy observations. We prove a strong universality result showing that Bayes-optimal performances are characterized by an equivalent Gaussian model with an effective…