Related papers: The evolution of lossy compression
The quintessential model-based reinforcement-learning agent iteratively refines its estimates or prior beliefs about the true underlying model of the environment. Recent empirical successes in model-based reinforcement learning with…
Shannon's information entropy measures of the uncertainty of an event's outcome. If learning about a system reflects a decrease in uncertainty, then a plausible intuition is that learning should be accompanied by a decrease in the entropy…
Inspired by the swarming or flocking of animal systems we study groups of agents moving in unbounded 2D space. Individual trajectories derive from a ``bottom-up'' principle: individuals reorient to maximise their future path entropy over…
Observations are an essential component of the simulation based studies on artificial-evolutionary systems (AES) by which entities are identified and their behavior is observed to uncover higher-level "emergent" phenomena. Because of the…
Biological and machine pattern recognition systems face a common challenge: Given sensory data about an unknown object, classify the object by comparing the sensory data with a library of internal representations stored in memory. In many…
Information theoretic analysis of large evolved programs produced by running genetic programming for up to a million generations has shown even functions as smooth and well behaved as floating point addition and multiplication loose entropy…
A major challenge of interdisciplinary description of complex system behaviour is whether real systems of higher complexity levels can be understood with at least the same degree of objective, "scientific" rigour and universality as…
A system comprised of an elastic solid and its response to an external random force sequence is shown to behave based on the principles of the theory of algorithmic complexity and randomness. The solid distorts the randomness of an input…
Highly nonlinear behavior of a system of discrete sites on a lattice is observed when a specific feedback loop is introduced into models employing coupled map lattices, quantum cellular automata, or the real-valued analogues of the latter.…
Chaitin's work, in its depth and breadth, encompasses many areas of scientific and philosophical interest. It helped establish the accepted mathematical concept of randomness, which in turn is the basis of tools that I have developed to…
The possibility that evolutionary forces -- together with a few fundamental factors such as thermodynamic constraints, specific computational features enabling information processing, and ecological processes -- might constrain the logic of…
Complex planning and scheduling problems have long been solved using various optimization or heuristic approaches. In recent years, imitation learning that aims to learn from expert demonstrations has been proposed as a viable alternative…
We explore how physical scale and population size shape the emergence of complex behaviors in open-ended ecological environments. In our setting, agents are unsupervised and have no explicit rewards or learning objectives but instead evolve…
Complex adaptive systems have been the subject of much recent attention. It is by now well-established that members (`agents') tend to self-segregate into opposing groups characterized by extreme behavior. However, while different social…
The ability of organisms to accurately sense their environment and respond accordingly is critical for evolutionary success. However, exactly how the sensory ability influences fitness is a topic of active research, while the necessity of a…
We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses: (1) learning in an individual human brain is hampered by the presence of effective…
In this article we use rate-distortion theory, a branch of information theory devoted to the problem of lossy compression, to shed light on an important problem in latent variable modeling of data: is there room to improve the model? One…
Sensory observations about the world are invariably ambiguous. Inference about the world's latent variables is thus an important computation for the brain. However, computational constraints limit the performance of these computations.…
Living systems operate far from equilibrium, yet few general frameworks provide global bounds on biological transients. In high-dimensional biological networks like ecosystems, long transients arise from the separate timescales of…
We show why the amount of information communicated between the past and future--the excess entropy--is not in general the amount of information stored in the present--the statistical complexity. This is a puzzle, and a long-standing one,…