Related papers: An Information-Theoretic Formalism for Multiscale …
Informational entropy is often identified as physical entropy. This is surprising because the two quantities are differently defined and furthermore the former is a subjective quantity while the latter is an objective one. We describe the…
It is not obvious what fraction of all the potential information residing in the molecules and structures of living systems is significant or meaningful to the system. Sets of random sequences or identically repeated sequences, for example,…
Network or graph structures are ubiquitous in the study of complex systems. Often, we are interested in complexity trends of these system as it evolves under some dynamic. An example might be looking at the complexity of a food web as…
We introduce a class of information measures based on group entropies, allowing us to describe the information-theoretical properties of complex systems. These entropic measures are nonadditive, and are mathematically deduced from a series…
To characterize the complex higher-order interactions among variables within a system, this study introduces a novel framework, termed System Information Decomposition (SID), aimed at decomposing the information entropy of variables into…
I discuss several aspects of information theory and its relationship to physics and neuroscience. The unifying thread of this somewhat chaotic essay is the concept of Kolmogorov or algorithmic complexity (Kolmogorov Complexity, for short).…
Predicting the evolution of a large system of units using its structure of interaction is a fundamental problem in complex system theory. And so is the problem of reconstructing the structure of interaction from temporal observations. Here,…
Among the various forms of reasoning studied in the context of artificial intelligence, qualitative reasoning makes it possible to infer new knowledge in the context of imprecise, incomplete information without numerical values. In this…
Shannon information theory provides various measures of so-called "syntactic information", which reflect the amount of statistical correlation between systems. In contrast, the concept of "semantic information" refers to those correlations…
To understand how an organisational structure relates to organisational behaviour is an interesting fundamental challenge in the area of organisation modelling. Specifications of organisational structure usually have a diagrammatic form…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
This chapter reviews four notions of system structure, three of which are contextual and classic (i.e. the complete computational structure linked to a state space model, the sparsity pattern of a transfer function, and the interconnection…
Information-theoretic measures such as relative entropy and correlation are extremely useful when modeling or analyzing the interaction of probabilistic systems. We survey the quantum generalization of 5 such measures and point out some of…
A qualitatively new, much more liberal and efficient organisation of science is proposed and justified, in connection with growing debate about further role and development of fundamental science. Although the key ideas can be explained…
Human beings possess the most sophisticated computational machinery in the known universe. We can understand language of rich descriptive power, and communicate in the same environment with astonishing clarity. Two of the many contributors…
Multi-scale structures are prevalent in both natural and artificial systems, as they can handle increasing complexity. Several terms are employed almost interchangeably across various application domains to refer to the multi-scale concept…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Information entropies give a genuine way to characterize quantitatively an incompatibility in quantum measurements. Together with the Shannon entropy, few families of parametrized entropies have found use in various questions. It is also…
Understanding how network structure constrains and enables information processing is a central problem in the statistical mechanics of interacting systems. Here we study random networks across the structural percolation transition and…
One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of…