Related papers: Architecture of Information
In this essay, a general case of information systems contains quantum information systems is considered. By presenting an algorithmic method a new kind of information topology is defined and considered. Continuous maps between two…
As our understanding of the mechanisms of brain function is enhanced, the value of insights gained from neuroscience to the development of AI algorithms deserves further consideration. Here, we draw parallels with an existing tree-based ANN…
Humans possess the capability to reason at an abstract level and to structure information into abstract categories, but the underlying neural processes have remained unknown. Experimental evidence has recently emerged for the organization…
We present a demonstration of the utility of NLP for aiding research into energetic materials and associated systems. The NLP method enables machine understanding of textual data, offering an automated route to knowledge discovery and…
The free energy principle (FEP) is a mathematical framework that describes how biological systems self-organize and survive in their environment. This principle provides insights on multiple scales, from high-level behavioral and cognitive…
Modelling the complex physics of the Interstellar Medium (ISM) in the context of large-scale numerical simulations is a challenging task. A number of methods have been proposed to embed a description of the ISM into different codes. We…
Self-organization is the autonomous assembly of a network of interacting components into a stable, organized pattern. This article shows that the process of self-assembly can be encoded in terms of evolutionary entropy, a statistical…
In the theory of special relativity, energy can be found in two forms: kinetic energy and rest mass. Potential energy of a body is actually stored under the form of rest mass, interaction energy too, temperature is not. Information acquired…
The neural mechanism of memory has a very close relation with the problem of representation in artificial intelligence. In this paper a computational model was proposed to simulate the network of neurons in brain and how they process…
The aim of this paper is to leverage the free-energy principle and its corollary process theory, active inference, to develop a generic, generalizable model of the representational capacities of living creatures; that is, a theory of…
Learning with physical systems is an emerging paradigm that seeks to harness the intrinsic nonlinear dynamics of physical substrates for learning. The impetus for a paradigm shift in how hardware is used for computational intelligence stems…
Architecture-based heat dissipation analyses allow us to reveal fundamental sources of inefficiency in a given processor and thereby provide us with road-maps to design less dissipative computing schemes independent of technology-base used…
Despite the remarkable success of large large-scale neural networks, we still lack unified notation for thinking about and describing their representational spaces. We lack methods to reliably describe how their representations are…
This paper is organized into four chapters. The first chapter observes how decoupling labor from ownership of the output produced by that labor is equivalent to building an engine that is incentivized to generate the maximum energy output,…
In communication networks structure and dynamics are tightly coupled. The structure controls the flow of information and is itself shaped by the dynamical process of information exchanged between nodes. In order to reconcile structure and…
Text classification is a primary task in natural language processing (NLP). Recently, graph neural networks (GNNs) have developed rapidly and been applied to text classification tasks. As a special kind of graph data, the tree has a simpler…
We develop a formal hierarchy of the expressive capacity of RNN architectures. The hierarchy is based on two formal properties: space complexity, which measures the RNN's memory, and rational recurrence, defined as whether the recurrent…
A unified combinatorial definition of the information content and entropy of different types of patterns, compatible with the traditional concepts of information and entropy, going beyond the limitations of Shannon information interpretable…
In recent years, artificial neural networks and their applications for large data sets have became a crucial part of scientific research. In this work, we implement the Multilayer Perceptron (MLP), which is a class of feedforward artificial…
In many scenarios, networks emerge endogenously as cognitive agents establish links in order to exchange information. Network formation has been widely studied in economics, but only on the basis of simplistic models that assume that the…