Related papers: Zipf's law, unbounded complexity and open-ended ev…
Why does Zipf's law give a good description of data from seemingly completely unrelated phenomena? Here it is argued that the reason is that they can all be described as outcomes of a ubiquitous random group division: the elements can be…
Although information theory has found success in disciplines, the literature on its applications to software evolution is limit. We are still missing artifacts that leverage the data and tooling available to measure how the information…
Biological systems reach hierarchical complexity that has no counterpart outside the realm of biology. Undoubtedly, biological entities obey the fundamental physical laws. Can today's physics provide an explanatory framework for…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
We show that the laws of Zipf and Benford, obeyed by scores of numerical data generated by many and diverse kinds of natural phenomena and human activity are related to the focal expression of a generalized thermodynamic structure. This…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
Background: Zipf's law and Heaps' law are two representatives of the scaling concepts, which play a significant role in the study of complexity science. The coexistence of the Zipf's law and the Heaps' law motivates different understandings…
The theory of evolution by natural selection cannot be used to evaluate the truth value of the following proposition: Through evolution, there exists at least one species that can adapt to any one given environment. To address this issue,…
Eukaryote genomes contain excessively introns, inter-genic and other non-genic sequences that appear to have no vital functional role or phenotype manifestation. Their existence, a long-standing puzzle, is viewed from the principle of…
Scale independence is a ubiquitous feature of complex systems which implies a highly skewed distribution of resources with no characteristic scale. Research has long focused on why systems as varied as protein networks, evolution and stock…
What is information? Is it physical? We argue that in a Bayesian theory the notion of information must be defined in terms of its effects on the beliefs of rational agents. Information is whatever constrains rational beliefs and therefore…
In information theory, Fisher information and Shannon information (entropy) are respectively used to quantify the uncertainty associated with the distribution modeling and the uncertainty in specifying the outcome of given variables. These…
In order to make a case for or against a trend in the evolution of complexity in biological evolution, complexity needs to be both rigorously defined and measurable. A recent information-theoretic (but intuitively evident) definition…
Understanding the innovation process, that is the underlying mechanisms through which novelties emerge, diffuse and trigger further novelties is undoubtedly of fundamental importance in many areas (biology, linguistics, social science and…
We inspect the deductive connection between the neural scaling law and Zipf's law -- two statements discussed in machine learning and quantitative linguistics. The neural scaling law describes how the cross entropy rate of a foundation…
We describe systems using Kauffman and similar networks. They are directed funct ioning networks consisting of finite number of nodes with finite number of discr ete states evaluated in synchronous mode of discrete time. In this paper we…
Evolution by Natural Selection is a process by which progeny inherit some properties from their progenitors with small variation. These properties are subject to Natural Selection and are called adaptive traits and carriers of the latter…
The notion of information pervades informal descriptions of biological systems, but formal treatments face the problem of defining a quantitative measure of information rooted in a concept of fitness, which is itself an elusive notion.…
In quantum mechanics, outcomes of measurements on a state have a probabilistic interpretation while the evolution of the state is treated deterministically. Here we show that one can also treat the evolution as being probabilistic in nature…
A family of information theoretic models of communication was introduced more than a decade ago to explain the origins of Zipf's law for word frequencies. The family is a based on a combination of two information theoretic principles:…