Related papers: Measures of Entropy from Data Using Infinitely Div…
Classification is a machine learning method used in many practical applications: text mining, handwritten character recognition, face recognition, pattern classification, scene labeling, computer vision, natural langage processing. A…
Entropy governs molecular self-assembly, phase transitions, and material stability, yet remains challenging to quantify and directly control in molecular systems. Here, we demonstrate that the computable information density (CID), a data…
Imaging systems are commonly described using resolution, contrast, and signal-to-noise ratio, but these quantities do not provide a general account of how physical transformations affect the flow of information. This paper introduces an…
Information-theoretic measures such as relative entropy and correlation are extremely useful when modeling or analyzing the interaction of probabilistic systems. We survey the quantum generalization of 5 such measures and point out some of…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
We discuss the properties of invariant measures corresponding to iterated function systems (IFSs) with place-dependent probabilities and compute their Renyi entropies, generalized dimensions, and multifractal spectra. It is shown that with…
Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
The similarity of local atomic environments is an important concept in many machine-learning techniques which find applications in computational chemistry and material science. Here, we present and discuss a connection between the…
Modern datasets across many disciplines increasingly consist of time-evolving, potentially infinite-dimensional random objects, such as dynamic functional data, which are naturally modeled in Hilbert spaces. In these settings,…
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated…
In estimating the complexity of objects, in particular of graphs, it is common practice to rely on graph- and information-theoretic measures. Here, using integer sequences with properties such as Borel normality, we explain how these…
Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…
Observational entropy is interpreted as the uncertainty an observer making measurements associates with a system. So far, properties that make such an interpretation possible rely on the assumption of ideal projective measurements. We show…
Administrative data are often easier to access as tabulated summaries than in the original format due to confidentiality concerns. Motivated by this practical feature, we propose a novel nonparametric density estimation method from…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
In this article, we present quantum algorithms for estimating von Neumann entropy and Renyi entropy, which are crucial physical and information-theoretical properties of a given quantum state $\rho$. Although there have been existing works…
Transfer entropy is a widely used measure for quantifying directed information flows in complex systems. While the challenges of estimating transfer entropy for continuous data are well known, it has two major shortcomings for data of…
This paper presents a command-line tool, called Entropia, that implements a family of conformance checking measures for process mining founded on the notion of entropy from information theory. The measures allow quantifying classical…
We prove that kernel covariance embeddings lead to information-theoretically perfect separation of distinct continuous probability distributions. In statistical terms, we establish that testing for the \emph{equality} of two non-atomic…