Related papers: Shannon Mutual Information Applied to Genetic Syst…
An existing conjecture states that the Shannon mutual information contained in the ground state wavefunction of conformally invariant quantum chains, on periodic lattices, has a leading finite-size scaling behavior that, similarly as the…
Despite its obvious relevance, meaning has been outside most theoretical approaches to information in biology. As a consequence, functional responses based on an appropriate interpretation of signals has been replaced by a probabilistic…
This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
This article serves as a brief introduction to the Shannon information theory. Concepts of information, Shannon entropy and channel capacity are mainly covered. All these concepts are developed in a totally combinatorial flavor. Some issues…
We consider the Moran model of population genetics with two types, mutation, and selection, and investigate the line of descent of a randomly-sampled individual from a contemporary population. We trace this ancestral line back into the…
In biology, information flows from the environment to the genome by the process of natural selection. But it has not been clear precisely what sort of information metric properly describes natural selection. Here, I show that Fisher…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
Natural selection can create information. In particular, because of the action of natural selection, we can often learn something about an environment by examining local organisms, and vice versa. For example, the characteristics of a…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
Although Shannon mutual information has been widely used, its effective calculation is often difficult for many practical problems, including those in neural population coding. Asymptotic formulas based on Fisher information sometimes…
There is evidence that biological synapses have only a fixed number of discrete weight states. Memory storage with such synapses behaves quite differently from synapses with unbounded, continuous weights as old memories are automatically…
The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown.…
We use the formalism of 'Maximum Principle of Shannon's Entropy' to derive the general power law distribution function, using what seems to be a reasonable physical assumption, namely, the demand of a constant mean "internal order"…
The naming game has become an archetype for linguistic evolution and mathematical social behavioral analysis. In the model presented here, there are $N$ individuals and $K$ words. Our contribution is developing a robust method that handles…
We investigate the evolutionary dynamics of a population structured in phenotype, subjected to trait dependent selection with a linearly moving optimum and an asexual mode of reproduction. Our model consists of a non-local and non-linear…
We consider a population model where individuals behave independently from each other and whose genealogy is described by a chronological tree called splitting tree. The individuals have i.i.d. (non-exponential) lifetime durations and give…
In this paper, we present a new multi-scale information content calculation method based on Shannon information (and Shannon entropy). The original method described by Claude E. Shannon and based on the logarithm of the probability of…
We consider a multi-type Moran model (in continuous time) with selection and type-dependent mutation. This paper is concerned with the evolution of genealogical information forward in time. For this purpose we define and analytically…
We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of…