Related papers: Information Geometry and Classical Cram\'{e}r-Rao …
We generalize the classical Fisher information metric on statistical models to $L^p$-metrics on various spaces of differential forms or group of diffeomorphisms. Using this new interpretation from information geometry, we derive several new…
Deep nonlinear models pose a challenge for fitting parameters due to lack of knowledge of the hidden layer and the potentially non-affine relation of the initial and observed layers. In the present work we investigate the use of information…
Shape inference is classically ill-posed, because it involves a map from the (2D) image domain to the (3D) world. Standard approaches regularize this problem by either assuming a prior on lighting and rendering or restricting the domain,…
Building upon a recent analysis of the measurement process in Hamiltonian mechanics, this article investigates the Bayesian epistemology of classical physics -- the landscape of accessible probability distributions over phase space. I prove…
Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different distributions (denoted as $\mu$ and $\mu'$, respectively). In this work, we give an…
In this paper, We develop the stratified de Rham theory on singular spaces using modern tools including derived geometry and stratified structures. This work unifies and extends the de Rham theory, Hodge theory, and deformation theory of…
Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and…
We derive a vector generalization of the curvature-corrected Cram\'er--Rao bound (CRB) in the nonasymptotic regime using a Hilbert space square-root embedding. Building on previous scalar results, we establish a \emph{directional} curvature…
We are studying Runge-Kutta methods along complex paths of integration from a geometric point of view. Thereby we derive special complex time grids, which applied to the problem of integrating a linear autonomous system of ordinary…
An abstract convergence theorem for a class of generalized descent methods that explicitly models relative errors is proved. The convergence theorem generalizes and unifies several recent abstract convergence theorems. It is applicable to…
The generalized Kullback-Leibler divergence (K-Ld) in Tsallis statistics [constrained by the additive duality of generalized statistics (dual generalized K-Ld)] is here reconciled with the theory of Bregman divergences for expectations…
We derive a closed form solution for the Kullback-Leibler divergence between two generalized gamma distributions. These notes are meant as a reference and provide a guided tour towards a result of practical interest that is rarely…
We derive the Kullback-Leibler divergence for the normal-gamma distribution and show that it is identical to the Bayesian complexity penalty for the univariate general linear model with conjugate priors. Based on this finding, we provide…
In this paper, we derive a useful lower bound for the Kullback-Leibler divergence (KL-divergence) based on the Hammersley-Chapman-Robbins bound (HCRB). The HCRB states that the variance of an estimator is bounded from below by the…
The entropy power inequality, which plays a fundamental role in information theory and probability, may be seen as an analogue of the Brunn-Minkowski inequality. Motivated by this connection to Convex Geometry, we survey various recent…
We study low-dimensional problems in topology and geometry via a study of contact and Cauchy-Riemann ($CR$) structures. A contact structure is called spherical if it admits a compatible spherical $CR$ structure. We will talk about spherical…
In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of 1) a weighted Mean Square Error (wMSE)…
This work proposes an algebraic model for classical information theory. We first give an algebraic model of probability theory. Information theoretic constructs are based on this model. In addition to theoretical insights provided by our…
Graph matching aims to establish correspondences between vertices of graphs such that both the node and edge attributes agree. Various learning-based methods were recently proposed for finding correspondences between image key points based…
The theories of strings and $D$-branes have motivated the development of non Abelian cohomology techniques in differential geometry, on the purpose to find a geometric interpretation of characteristic classes. The spaces studied here, like…