Related papers: Fisher information matrix of binary time series

Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce…

Famously, the quantum Fisher information -- the maximum Fisher information over all physical measurements -- is additive for independent copies of a system and the optimal measurement acts locally. We are left to wonder: does the same hold…

Model merging has emerged as a practical approach to combine capabilities of specialized large language models (LLMs) without additional training. In the Long-to-Short (L2S) scenario, merging a base model with a long-chain-of-thought…

Generalized Linear Models (GLMs) and Single Index Models (SIMs) provide powerful generalizations of linear regression, where the target variable is assumed to be a (possibly unknown) 1-dimensional function of a linear predictor. In general,…

Firth-type logistic regression has become a standard approach for the analysis of binary outcomes with small samples. Whereas it reduces the bias in maximum likelihood estimates of coefficients, bias towards 1/2 is introduced in the…

We consider the processing of statistical samples $X\sim P_\theta$ by a channel $p(y|x)$, and characterize how the statistical information from the samples for estimating the parameter $\theta\in\mathbb{R}^d$ can scale with the mutual…

In the field of statistical learning and data analysis, estimating precision matrices (i.e., the inverse of covariance matrices) is a critical task, particularly for understanding dependency structures among variables. However, traditional…

We derive general upper bounds to pointwise mutual information in terms of stochastic Fisher information and show these bounds average to known results in the literature for bounds to mutual information in terms of Fisher information. These…

Expected Fisher information can be found a priori and as a result its inverse is the primary variance approximation used in the design of experiments. This is in contrast to the common claim that the inverse of observed Fisher information…

We present two extended forms of Fisher information that fit well in the context of nonextensive thermostatistics. We show that there exists an interplay between these generalized Fisher information, the generalized $q$-Gaussian…

In stochastic optimization, the population risk is generally approximated by the empirical risk. However, in the large-scale setting, minimization of the empirical risk may be computationally restrictive. In this paper, we design an…

This paper proposes an alternative approach for constructing invariant Jeffreys prior distributions tailored for hierarchical or multilevel models. In particular, our proposal is based on a flexible decomposition of the Fisher information…

The analysis of non-real-valued data, such as binary time series, has attracted great interest in recent years. This manuscript proposes a post-selection estimator for estimating the coefficient matrices of a high-dimensional generalized…

The analysis of longitudinal, heterogeneous or unbalanced clustered data is of primary importance to a wide range of applications. The Linear Mixed Model (LMM) is a popular and flexible extension of the linear model specifically designed…

The effective Fisher matrix method recently introduced by Cho et al. is a semi-analytic approach to the Fisher matrix, in which a local overlap surface is fitted by using a quadratic fitting function. Mathematically, the effective Fisher…

In this paper, we analyze the impact of compressed sensing with complex random matrices on Fisher information and the Cram\'{e}r-Rao Bound (CRB) for estimating unknown parameters in the mean value function of a complex multivariate normal…

Approximations are commonly employed in realistic applications of scientific Bayesian inference, often due to convenience if not necessity. In the field of gravitational-wave (GW) data analysis, fast-to-evaluate but approximate waveform…

Data privacy concerns have led to the growing interest in synthetic data, which strives to preserve the statistical properties of the original dataset while ensuring privacy by excluding real records. Recent advances in deep neural networks…

Mimicking the maximum likelihood estimator, we construct first order Cramer-Rao efficient and explicitly computable estimators for the scale parameter $\sigma^2$ in the model $Z_{i,n}=\sigma n^{-\beta}X_i+Y_i,i=1,\ldots,n,\beta>0$ with…

This paper establishes bounds on the performance of empirical risk minimization for large-dimensional linear regression. We generalize existing results by allowing the data to be dependent and heavy-tailed. The analysis covers both the…