Related papers: Entropy and inference, revisited
In this note we study some properties of topological entropy for non-compact non-metrizable spaces. We prove that if a uniformly continuous self-map $f$ of a uniform space has topological shadowing property then the map $f$ has positive…
In this note we study some properties of topological entropy for noncompact non-metrizable spaces.
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
We demonstrate that the functional form of the likelihood contains a sufficient amount of information for constructing a prior for the unknown parameters. We develop a four-step algorithm by invoking the information entropy as the measure…
Classifications organize entities into categories that identify similarities within a category and discern dissimilarities among categories, and they powerfully classify information in support of analysis. We propose a new classification…
Entropy measures of probability distributions are widely used measures in ecology, biology, genetics, and in other fields, to quantify species diversity of a community. Unfortunately, entropy-based diversity indices, or diversity indices…
We study the compact embedding between smoothness Morrey spaces on bounded domains and characterise its entropy numbers. Here we discover a new phenomenon when the difference of smoothness parameters in the source and target spaces is…
We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha > 0$ (i.e. when the true underlying…
We generalize the definition of topological entropy given by Adler, Konheim, and McAndrew (AKM) for piecewise continuous self-maps defined on a compact interval (pc-maps). For this notion of entropy, we prove that the properties of the…
We describe and analyze some Monte Carlo methods for manifolds in Euclidean space defined by equality and inequality constraints. First, we give an MCMC sampler for probability distributions defined by un-normalized densities on such…
We study the continuity property of the generalized entropy as a function of the underlying probability distribution, defined with an action space and a loss function, and use this property to answer the basic questions in statistical…
We study how the training data distribution affects confidence and performance in image classification models. We introduce Embedding Density, a model-agnostic framework that estimates prediction confidence by measuring the distance of test…
Deep Ensembles, as a type of Bayesian Neural Networks, can be used to estimate uncertainty on the prediction of multiple neural networks by collecting votes from each network and computing the difference in those predictions. In this paper,…
There is a rich literature on Bayesian methods for density estimation, which characterize the unknown density as a mixture of kernels. Such methods have advantages in terms of providing uncertainty quantification in estimation, while being…
The aim of this paper is to discuss new results concerning some kinds of parametric extended entropies and divergences. As a result of our studies for mathematical properties on entropy and divergence, we give new bounds for the Tsallis…
Given a sample of independent and identically distributed random variables, a novel nonparametric maximum entropy method is presented to estimate the underlying continuous univariate probability density function (pdf). Estimates are found…
We provide a general framework for learning diffusion bridges that transport prior to target distributions. It includes existing diffusion models for generative modeling, but also underdamped versions with degenerate diffusion matrices,…
Prior Networks are a recently developed class of models which yield interpretable measures of uncertainty and have been shown to outperform state-of-the-art ensemble approaches on a range of tasks. They can also be used to distill an…
Entropy notions for $\varepsilon$-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which…
Entropies associated with spatial subsystems in conventional local quantum field theories are typically divergent when the spatial regions have boundaries. However, in certain linear combinations of the entropies for various subsystems,…