Related papers: Information Geometry and Classical Cram\'{e}r-Rao …
Learning and reasoning over graphs is increasingly done by means of probabilistic models, e.g. exponential random graph models, graph embedding models, and graph neural networks. When graphs are modeling relations between people, however,…
There are many information and divergence measures exist in the literature on information theory and statistics. The most famous among them are Kullback-Leiber's (1951)relative information and Jeffreys (1946) J-divergence, Information…
A well-known technique in estimating probabilities of rare events in general and in information theory in particular (used, e.g., in the sphere-packing bound), is that of finding a reference probability measure under which the event of…
We present a technique for constructing random fields from a set of training samples. The learning paradigm builds increasingly complex fields by allowing potential functions, or features, that are supported by increasingly large subgraphs.…
Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic…
Explicit classical states achieving maximal $f$-divergence are given, allowing for a simple proof of Matsumoto's Theorem, and the systematic extension of any inequality between classical $f$-divergences to quantum $f$-divergences. Our…
This paper deals with Cram\'er-Rao inequalities in the context of nonextensive statistics and in estimation theory. It gives characterizations of generalized q-Gaussian distributions, and introduces generalized versions of Fisher…
Information geometry is a mathematical framework that elucidates the manifold structure of the probability distribution space (p-space), providing a systematic approach to transforming probability distributions (PDs). In this study, we…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
We reduce to various absolute parallelisms, namely to certain {e}-structures on manifolds of dimensions 7, 6, 5, the biholomorphic equivalence problem or the intrinsic CR equivalence problem for generic submanifolds M^5 in C^4 of CR…
The aim of these notes is to present an accessible overview of some topics in classical algebraic geometry which have applications to aspects of discrete integrable systems. Precisely, we focus on surface theory on the algebraic geometry…
Many natural and social science systems are described using probability distributions over elements that are related to each other: for instance, occupations with shared skills or species with similar traits. Standard information theory…
We study two kinds of curvature invariants of Riemannian manifold equip\-ped with a complex distribution $D$ (for example, a CR-submanifold of an almost Hermitian manifold) related to sets of pairwise orthogonal subspaces of the…
Information geometry is concerned with the application of differential geometry concepts in the study of the parametric spaces of statistical models. When the random variables are independent and identically distributed, the underlying…
Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari-Chentsov tensor. In statistics, the notion of sufficient statistic expresses the…
A geometrical formulation of estimation theory for finite-dimensional $C^{\star}$-algebras is presented. This formulation allows to deal with the classical and quantum case in a single, unifying mathematical framework. The derivation of the…
We prove the algebraicity of smooth $CR$-mappings between algebraic Cauchy--Riemann manifolds. A generalization of separate algebraicity principle is established.
Estimating the Kullback-Leibler (KL) divergence between random variables is a fundamental problem in statistical analysis. For continuous random variables, traditional information-theoretic estimators scale poorly with dimension and/or…
Classical integral geometry takes place in Euclidean space, but one can attempt to imitate it in any other metric space. In particular, one can attempt this in R^n equipped with the metric derived from the p-norm. This has, in effect, been…
In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of (1) a weighted Mean Square Error (wMSE)…