Related papers: Architecture of Information
Humans communicate using systems of interconnected stimuli or concepts -- from language and music to literature and science -- yet it remains unclear how, if at all, the structure of these networks supports the communication of information.…
To solve more complex things, computer systems becomes more and more complex. It becomes harder to be handled manually for various conditions and unknown new conditions in advance. This situation urgently requires the development of…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Modern buildings encompass complex dynamics of multiple electrical, mechanical, and control systems. One of the biggest hurdles in applying conventional model-based optimization and control methods to building energy management is the huge…
Free-energy landscape theory is often used to describe complex molecular systems. Here, a microscopic description of water structure and dynamics based on configuration-space-networks and molecular dynamics simulations of the TIP4P/2005…
Complex systems produce high-dimensional signals that lack macroscopic variables analogous to entropy, temperature, or free energy. This work introduces a thermoinformational formulation that derives entropy, internal energy, temperature,…
This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in…
The minimization of Gibbs free energy is based on the changes in work and free energy that occur in a physical or chemical system. The maximization of mutual information, the capacity, of a noisy channel is determined based on the marginal…
This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…
Artificial neural networks (ANNs), which are inspired by the brain, are a central pillar in the ongoing breakthrough in artificial intelligence. In recent years, researchers have examined mechanical implementations of ANNs, denoted as…
To make progress in science, we often build abstract representations of physical systems that meaningfully encode information about the systems. The representations learnt by most current machine learning techniques reflect statistical…
Introduction: In contrast to current AI technology, natural intelligence -- the kind of autonomous intelligence that is realized in the brains of animals and humans to attain in their natural environment goals defined by a repertoire of…
We develop an axiomatic reconstruction of thermodynamics based entirely on two primitive components: a description of what aspects of a system are observed and a reference measure that encodes the underlying descriptive convention. These…
We introduce a unified theoretical framework for the rigorous analysis and systematic construction of deep neural networks (DNNs). This framework addresses a gap in existing theory by explicitly modeling the structure of tensor operations…
We outline a possible theoretical framework for the quantitative modeling of networked embodied cognitive systems. We notice that: 1) information self structuring through sensory-motor coordination does not deterministically occur in Rn…
We introduce a simple initial working system in which relations (such as part-whole) are directly represented via an architecture with operating and learning rules fundamentally distinct from standard artificial neural network methods.…
Biological neural networks are characterized by their high degree of plasticity, a core property that enables the remarkable adaptability of natural organisms. Importantly, this ability affects both the synaptic strength and the topology of…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
Hypergraphs are a powerful abstraction for representing higher-order interactions between entities of interest. To exploit these relationships in making downstream predictions, a variety of hypergraph neural network architectures have…
Associative networks theory is increasingly providing tools to interpret update rules of artificial neural networks. At the same time, deriving neural learning rules from a solid theory remains a fundamental challenge. We make some steps in…