Related papers: t-Entropy: A New Measure of Uncertainty with Some …

The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs be obtained from the data sample itself. We propose a semi-parametric…

Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the popular cross-entropy…

Although some information-theoretic measures of uncertainty or granularity have been proposed in rough set theory, these measures are only dependent on the underlying partition and the cardinality of the universe, independent of the lower…

The investor is interested in the expected return and he is also concerned about the risk and the uncertainty assumed by the investment. One of the most popular concepts used to measure the risk and the uncertainty is the variance and/or…

Heisenberg's uncertainty principle has recently led to general measurement uncertainty relations for quantum systems: incompatible observables can be measured jointly or in sequence only with some unavoidable approximation, which can be…

The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…

Conventional semi-supervised contrastive learning methods assign pseudo-labels only to samples whose highest predicted class probability exceeds a predefined threshold, and then perform supervised contrastive learning using those selected…

In the last few years, many different performance measures have been introduced to overcome the weakness of the most natural metric, the Accuracy. Among them, Matthews Correlation Coefficient has recently gained popularity among researchers…

Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond…

We introduce a variant of the R\'enyi entropy definition that aligns it with the well-known H\"older mean: in the new formulation, the r-th order R\'enyi Entropy is the logarithm of the inverse of the r-th order H\"older mean. This brings…

The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at…

Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…

Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability…

Similarity-sensitive entropy measures the uncertainty of a probability law relative to a similarity kernel that encodes the distinguishability between states. We develop a measure-theoretic treatment covering both finite similarity matrices…

Large language models (LLMs) have transformed natural language processing, but their reliable deployment requires effective uncertainty quantification (UQ). Existing UQ methods are often heuristic and lack a probabilistic interpretation.…

This thesis develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…

Entropic uncertainty relations are powerful tools, especially in quantum cryptography. They typically bound the amount of uncertainty a third-party adversary may hold on a measurement outcome as a result of the measurement overlap. However,…

Quantum entropy is an important measure for describing the uncertainty of a quantum state, more uncertainty in subsystems implies stronger quantum entanglement between subsystems. Our goal in this work is to quantify bipartite entanglement…

Applying a machine learning model for decision-making in the real world requires to distinguish what the model knows from what it does not. A critical factor in assessing the knowledge of a model is to quantify its predictive uncertainty.…

In a paper [8] the authors classify entropy into three categories, as a thermodynamics quantity, as a measure of information production, and as a means of statistical inference. An entropy measure introduced by Mathai falls into the second…