Related papers: Entropy and Diversity: The Axiomatic Approach
Information theory plays a central role in establishing fundamental limits on what any learning or estimation algorithm can -- and cannot -- achieve, regardless of computational power. In this chapter, we provide an introduction to these…
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
The concept of entropy in statistical physics is related to the existence of irreversible macroscopic processes. In this work, we explore a recently introduced entropy formula for a class of stochastic processes with more than one absorbing…
Much of uncertainty quantification to date has focused on determining the effect of variables modeled probabilistically, and with a known distribution, on some physical or engineering system. We develop methods to obtain information on the…
In recent years, biodiversity measures have gained prominence as essential tools for ecological and environmental assessments, particularly in the context of increasingly complex and large-scale datasets. We provide a comprehensive review…
We propose a new way to measure the balance between freedom and coherence in a dynamical system and a new measure of its internal variability. Based on the concept of entropy and ideas from neuroscience and information theory, we define…
The problem of defining and studying complexity of a time series has interested people for years. In the context of dynamical systems, Grassberger has suggested that a slow approach of the entropy to its extensive asymptotic limit is a sign…
In this paper entropy based methods are compared and used to measure structural diversity of an ensemble of 21 classifiers. This measure is mostly applied in ecology, whereby species counts are used as a measure of diversity. The measures…
There is an overall perception of increased interdisciplinarity in science, but this is difficult to confirm quantitatively owing to the lack of adequate methods to evaluate subjective phenomena. This is no different from the difficulties…
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence…
Although some information-theoretic measures of uncertainty or granularity have been proposed in rough set theory, these measures are only dependent on the underlying partition and the cardinality of the universe, independent of the lower…
Information-theoretic quantities, such as entropy, are used to quantify the amount of information a given variable provides. Entropies can be used together to compute the mutual information, which quantifies the amount of information two…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
Theoretical physics is the search for simple and universal mathematical descriptions of the natural world. In contrast, much of modern biology is an exploration of the complexity and diversity of life. For many, this contrast is prima facie…
It is well known that equilibrium in a thermodynamic system results from a competition or balance between lowering the energy and increasing the entropy, or at least the product of the temperature and entropy. This is remarkably similar to…
This book chapter attempts to counter anxieties in the humanities and social science about the role of big data in research by focusing on approaches which, by being firmly grounded in the traditional values of disciplines, enhance existing…
Mathematical oncology is an interdisciplinary research field where the mathematical sciences meet cancer research. Being situated at the intersection of these two fields makes mathematical oncology highly dynamic, as practicing researchers…
Markov categories are a novel framework to describe and treat problems in probability and information theory. In this work we combine the categorical formalism with the traditional quantitative notions of entropy, mutual information, and…
The notion of metric entropy dimension is introduced to measure the complexity of entropy zero dynamical systems. For measure preserving systems, we define entropy dimension via the dimension of entropy generating sequences. This…
Approximation theory is concerned with the ability to approximate functions by simpler and more easily calculated functions. The first question we ask in approximation theory concerns the {\it possibility of approximation}. Is the given…