Related papers: Deformed Statistics Kullback-Leibler Divergence Mi…
We analyzed the effect of the deviation of the exact distribution of the p-values from the uniform distribution on the Kolmogorov-Smirnov (K-S) test that was implemented as the second-level randomness test. We derived an inequality that…
We address the problem of detecting a change in the distribution of a high-dimensional multivariate normal time series. Assuming that the post-change parameters are unknown and estimated using a window of historical data, we extend the…
This paper introduces an estimator of the relative directed distance between an estimated model and the true model, based on the Kulback-Leibler divergence and is motivated by the generalized information criterion proposed by Konishi and…
In optimization, the natural gradient method is well-known for likelihood maximization. The method uses the Kullback-Leibler divergence, corresponding infinitesimally to the Fisher-Rao metric, which is pulled back to the parameter space of…
This paper continues our survey about the mean-field derivation of the two-dimensional signal-dependent Keller-Segel system studied in [1]. Therefore, we consider the same system of moderately interacting particles as before. The difference…
Unsupervised learning of disentangled representations involves uncovering of different factors of variations that contribute to the data generation process. Total correlation penalization has been a key component in recent methods towards…
The Lutz-Kelker correction is intended to give an unbiased estimate for stellar parallaxes and magnitudes, but it is shown explicitly that it does not. This paradox results from the application of an argument about sample statistics to the…
Deep nonlinear models pose a challenge for fitting parameters due to lack of knowledge of the hidden layer and the potentially non-affine relation of the initial and observed layers. In the present work we investigate the use of information…
This paper presents a general methodology for deriving information-theoretic generalization bounds for learning algorithms. The main technical tool is a probabilistic decorrelation lemma based on a change of measure and a relaxation of…
We investigate the cumulative Tsallis entropy, an information measure recently introduced as a cumulative version of the classical Tsallis differential entropy, which is itself a generalization of the Boltzmann-Gibbs statistics. This…
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine…
Reduced k-means clustering is a method for clustering objects in a low-dimensional subspace. The advantage of this method is that both clustering of objects and low-dimensional subspace reflecting the cluster structure are simultaneously…
The relationship between the Bayesian approach and the minimum description length approach is established. We sharpen and clarify the general modeling principles MDL and MML, abstracted as the ideal MDL principle and defined from Bayes's…
A generalized Kullback-Leibler relative entropy is introduced starting with the symmetric Jackson derivative of the generalized overlap between two probability distributions. The generalization retains much of the structure possessed by the…
This paper concerns the approximation of probability measures on $\mathbf{R}^d$ with respect to the Kullback-Leibler divergence. Given an admissible target measure, we show the existence of the best approximation, with respect to this…
Black box variational inference (BBVI) with reparameterization gradients triggered the exploration of divergence measures other than the Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we view BBVI with…
This paper deals with the problem of estimating predictive densities of a matrix-variate normal distribution with known covariance matrix. Our main aim is to establish some Bayesian predictive densities related to matricial shrinkage…
In this short note, we give the convergence analysis of the policy in the recent famous policy mirror descent (PMD). We mainly consider the unregularized setting following [11] with generalized Bregman divergence. The difference is that we…
The size of the effect of the difference in two groups with respect to a variable of interest may be estimated by the classical Cohen's $d$. A recently proposed generalized estimator allows conditioning on further independent variables…
We approximate a given rational spectral density by one that is consistent with prescribed second-order statistics. Such an approximation is obtained by minimizing a suitable distance from the given spectrum and under the constraints…