Related papers: Is Complexity an Illusion?
Cooperation is a persistent behavioral pattern of entities pooling and sharing resources. Its ubiquity in nature poses a conundrum. Whenever two entities cooperate, one must willingly relinquish something of value to the other. Why is this…
A fundamental trait of intelligence is the ability to achieve goals in the face of novel circumstances, such as making decisions from new action choices. However, standard reinforcement learning assumes a fixed set of actions and requires…
In many contexts, simpler models are preferable to more complex models and the control of this model complexity is the goal for many methods in machine learning such as regularization, hyperparameter tuning and architecture design. In deep…
Biological systems are generally complicated and/or complex. In the former approach, one sets up a model with a large number of parameters to describe the system in detail. The latter approach focuses on understanding the universal aspects…
Learning robust and generalisable abstractions from high-dimensional input data is a central challenge in machine learning and its applications to high-energy physics (HEP). Solutions of lower functional complexity are known to produce…
While we have intuitive notions of structure and complexity, the formalization of this intuition is non-trivial. The statistical complexity is a popular candidate. It is based on the idea that the complexity of a process can be quantified…
Self-organization in complex systems is a process in which randomness is reduced and emergent structures appear that allow the system to function in a more competitive way with other states of the system or with other systems. It occurs…
We develop a general formalism for representing and understanding structure in complex systems. In our view, structure is the totality of relationships among a system's components, and these relationships can be quantified using information…
One of the presuppositions of science since the times of Galileo, Newton, Laplace, and Descartes has been the predictability of the world. This idea has strongly influenced scientific and technological models. However, in recent decades,…
One of the principal scientific challenges in deep learning is explaining generalization, i.e., why the particular way the community now trains networks to achieve small training error also leads to small error on held-out data from the…
The general concept of symmetry is realized in manifold ways in different realms of reality, such as plants, animals, minerals, mathematical objects or human artefacts in literature, fine arts and society. In order to arrive at a common…
At its core, abstraction is the process of generalizing from specific instances to broader concepts or models, with the primary objective of reducing complexity while preserving properties essential to the intended purpose. It is…
Very large deep learning models trained using gradient descent are remarkably resistant to memorization given their huge capacity, but are at the same time capable of fitting large datasets of pure noise. Here methods are introduced by…
We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization. Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of…
Deep neural networks, when optimized with sufficient data, provide accurate representations of high-dimensional functions; in contrast, function approximation techniques that have predominated in scientific computing do not scale well with…
The compactness phenomenon is one of the featured aspects of structuralism in mathematics. In simple and broad words, a compactness property holds in a structure if a related property is satisfied by sufficiently many substructures of that…
Constraint propagation is one of the techniques central to the success of constraint programming. To reduce search, fast algorithms associated with each constraint prune the domains of variables. With global (or non-binary) constraints, the…
The friendship paradox states that your friends have on average more friends than you have. Does the paradox "hold" for other individual characteristics like income or happiness? To address this question, we generalize the friendship…
The idea behind object-centric representation learning is that natural scenes can better be modeled as compositions of objects and their relations as opposed to distributed representations. This inductive bias can be injected into neural…
Recent work in cognitive neuroscience has focused on analyzing the brain as a network, rather than as a collection of independent regions. Prior studies taking this approach have found that individual differences in the degree of modularity…