Related papers: Information driven self-organization of complex ro…
Information theory provides a useful tool to understand the evolution of complex nonlinear systems and their sustainability. In particular, Fisher Information (FI) has been evoked as a useful measure of sustainability and the variability of…
A neural network with a learning algorithm optimized by information theory entropic dynamics is used to build an agent dubbed Homo Entropicus. The algorithm can be described at a macroscopic level in terms of aggregate variables…
This paper presents a novel approach to machine learning algorithm design based on information theory, specifically mutual information (MI). We propose a framework for learning and representing functional relationships in data using…
Adaptive dynamical systems arise in a multitude of contexts, e.g., optimization, control, communications, signal processing, and machine learning. A precise characterization of their fundamental limitations is therefore of paramount…
Part 1 has studied the conversion of observed random process with its hidden information to related dynamic process, applying entropy functional measure (EF) of the random process and path functional information measure (IPF) of the dynamic…
Maximum entropy principle identifies forces conjugated to observables and the thermodynamic relations between them, independent upon their underlying mechanistic details. For data about state distributions or transition statistics, the…
Information engines can use structured environments as a resource to generate work by randomizing ordered inputs and leveraging the increased Shannon entropy to transfer energy from a thermal reservoir to a work reservoir. We give a broadly…
Synthetic data becomes crucial for large language model training, but its effectiveness is highly inconsistent. We provide an information-theoretic account of this inconsistency: synthetic data improves a model only when the…
A fundamental question in the conjunction of information theory, biophysics, bioinformatics and thermodynamics relates to the principles and processes that guide the development of natural intelligence in natural environments where…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
This paper revisits Brownian motion from the perspective of Information Theory, aiming to explore the connections between Information Theory, Thermodynamics, and Complex Science. First, we propose a single-particle discrete Brownian motion…
Observational learning is a promising approach to enable people without expertise in programming to transfer skills to robots in a user-friendly manner, since it mirrors how humans learn new behaviors by observing others. Many existing…
The apparent dichotomy between information-processing and dynamical approaches to complexity science forces researchers to choose between two diverging sets of tools and explanations, creating conflict and often hindering scientific…
The abundance of process operating data in modern industries, along with the rapid advancement of learning techniques, has led to a paradigm shift towards data-centric analysis and control. However, integrating machine learning with control…
This paper is concerned with designing self-driven fitness functions for Embedded Evolutionary Robotics. The proposed approach considers the entropy of the sensori-motor stream generated by the robot controller. This entropy is computed…
Using mobile phone records and information theory measures, our daily lives have been recently shown to follow strict statistical regularities, and our movement patterns are to a large extent predictable. Here, we apply entropy and…
This paper considers optimal control of dynamical systems which are represented by nonlinear stochastic differential equations. It is well-known that the optimal control policy for this problem can be obtained as a function of a value…
Motion planning for autonomous robots in dynamic environments poses numerous challenges due to uncertainties in the robot's dynamics and interaction with other agents. Sampling-based MPC approaches, such as Model Predictive Path Integral…
In this paper, we view a policy or plan as a transition system over a space of information states that reflect a robot's or other observer's perspective based on limited sensing, memory, computation, and actuation. Regardless of whether…
Measuring the average information that is necessary to describe the behaviour of a dynamical system leads to a generalization of the Kolmogorov-Sinai entropy. This is particularly interesting when the system has null entropy and the…