Related papers: Extropy-Based Generalized Divergence and Similarit…
Vector similarity measures play a fundamental role in various fields, including machine learning, natural language processing, information retrieval, and data mining. These measures quantify the closeness between two vectors in a…
The most popular approach in extreme value statistics is the modelling of threshold exceedances using the asymptotically motivated generalised Pareto distribution. This approach involves the selection of a high threshold above which the…
Competing and Complementary risk (CCR) problems are often modelled using a class of distributions of the maximum, or minimum, of a random number of i.i.d. random variables; we call this class the CCR class of distributions. While the CCR…
We study data processing inequalities that are derived from a certain class of generalized information measures, where a series of convex functions and multiplicative likelihood ratios are nested alternately. While these information…
In this work, we propose to study the global geometrical properties of generative models. We introduce a new Riemannian metric to assess the similarity between any two data points. Importantly, our metric is agnostic to the parametrization…
This paper extends the asymmetric Kullback-Leibler divergence and symmetric Jensen-Shannon divergence from two probability measures to the case of two sets of probability measures. We establish some fundamental properties of these…
Bayesian methods and their implementations by means of sophisticated Monte Carlo techniques, such as Markov chain Monte Carlo (MCMC) and particle filters, have become very popular in signal processing over the last years. However, in many…
A measure of similarity between text embeddings can be considered adequate only if it adheres to the human perception of similarity between texts. In this paper, we introduce the distance-to-distance ratio (DDR), a novel measure of…
The notion of weighted Renyi's entropy for truncated random variables has recently been proposed in the information-theoretic literature. In this paper, we introduce a generalized measure of it for double truncated distribution, namely…
We show how generalized Gibbs-Shannon entropies can provide new insights on the statistical properties of texts. The universal distribution of word frequencies (Zipf's law) implies that the generalized entropies, computed at the word level,…
In many contexts Gaussian Mixtures (GM) are used to approximate probability distributions, possibly time-varying. In some applications the number of GM components exponentially increases over time, and reduction procedures are required to…
Measuring the correlation (association) between two random variables is one of the important goals in statistical applications. In the literature, the covariance between two random variables is a widely used criterion in measuring the…
We establish analogues for trees of results relating the density of a set $E \subset \mathbb{N}$, the density of its set of popular differences, and the structure of $E$. To obtain our results, we formalise a correspondence principle of…
In this paper, we propose a novel association measure for longitudinal studies based on the traditional definition of relative risk. In a Markovian fashion, such a proposal takes into account the information content regarding the previous…
Understanding geometric properties of natural language processing models' latent spaces allows the manipulation of these properties for improved performance on downstream tasks. One such property is the amount of data spread in a model's…
The Gaussian theory of errors has been generalized to situations, where the Gaussian distribution and, hence, the Gaussian rules of error propagation are inadequate. The generalizations are based on Bayes' theorem and a suitable measure.…
The aim of this work is to establish that two recently published projection theorems, one dealing with a parametric generalization of relative entropy and another dealing with R\'{e}nyi divergence, are equivalent under a correspondence on…
We propose a novel kernel-based nonparametric two-sample test, employing the combined use of kernel mean and kernel covariance embedding. Our test builds on recent results showing how such combined embeddings map distinct probability…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
The probability density function (PDF) plays a central role in statistical and machine learning modeling. Real-world data often deviates from Gaussian assumptions, exhibiting skewness and exponential decay. To evaluate how well different…