Related papers: Stochastic comparisons, differential entropy and v…
We study distributional similarity measures for the purpose of improving probability estimation for unseen cooccurrences. Our contributions are three-fold: an empirical comparison of a broad range of measures; a classification of similarity…
We review various methods used to estimate uncertainties in quantum correlation functions, such as parton distribution functions (PDFs). Using a toy model of a PDF, we compare the uncertainty estimates yielded by the traditional Hessian and…
Non-deterministic measurements are common in real-world scenarios: the performance of a stochastic optimization algorithm or the total reward of a reinforcement learning agent in a chaotic environment are just two examples in which…
For a random variable we can define a variational relationship with practical physical meaning as dI=dbar(x)-bar(dx), where I is called as uncertainty measurement. With the help of a generalized definition of expectation,…
We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…
Sensitivity analysis of a numerical model, for instance simulating physical phenomena, is useful to quantify the influence of the inputs on the model responses. This paper proposes a new sensitivity index, based upon the modification of the…
In this paper, we consider distributed maximum likelihood estimation (MLE) with dependent quantized data under the assumption that the structure of the joint probability density function (pdf) is known, but it contains unknown deterministic…
The Varentropy is a measure of the variability of the information content of random vector and it is invariant under affine transformations. We introduce the statistical estimate of varentropy of random vector based on the nearest neighbor…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
Supersonic turbulence occurs in many environments, particularly in astrophysics. In the crucial case of isothermal turbulence, the probability density function (PDF) of the logarithmic density, $s$, is well measured, but a theoretical…
The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…
We study local power fluctuations in numerical simulations of stationary, homogeneous, isotropic turbulence in two and three dimensions with Gaussian forcing. Due to the near-Gaussianity of the one-point velocity distribution, the…
We use the method of Maximum (relative) Entropy to process information in the form of observed data and moment constraints. The generic "canonical" form of the posterior distribution for the problem of simultaneous updating with data and…
The statistical duality of distributions is a powerful tool for statistical inferences. In the paper the statistical duality of Laplace distribution is discussed. As shown the confidence density of the parameter of this distribution is…
Following [1], the aim of this paper is to analyze the relative weighted entropy involving the central moments weight functions. We compare the standard relative entropy with the weighted case in two particular forms of Gaussian…
This paper outlines an approach to the approximation of probability density functions by quadratic forms of weighted orthonormal basis functions with positive semi-definite Hermitian matrices of unit trace. Such matrices are called…
We introduce novel information-entropic variables -- a Point Divergence Gain (${\Omega}^{(l \rightarrow m)}_\alpha$), a Point Divergence Gain Entropy ($I_\alpha$), and a Point Divergence Gain Entropy Density ($P_\alpha$) -- which are…
We show one possible dynamical approach to the study of the distribution of prime numbers. Our approach is based on two complexity methods, the Computable Information Content and the Entropy Information Gain, looking for analogies between…
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated…