Related papers: Component systems: do null models explain everythi…
Inspired from human cognition, machine learning systems are gradually revealing advantages of sparser and more modular architectures. Recent work demonstrates that not only do some modular architectures generalize well, but they also lead…
Model sets (also called cut and project sets) are generalizations of lattices, and multi-component model sets are generalizations of lattices with colourings. In this paper, we study self-similarities of multi-component model sets. The main…
Compositionality is thought to be a key component of language, and various compositional benchmarks have been developed to empirically probe the compositional generalization of existing sequence processing models. These benchmarks often…
This essay advocates the view that any problem that has a meaningful empirical content, can be formulated in constructive, more definitely, finite terms. We consider combinatorial models of dynamical systems and approaches to statistical…
Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems. Although modern models are trained on massive datasets, they still cover only a tiny fraction of the…
Many theories are formulated as constrained systems. We provide a mechanism that explains the origin of physical states of a constrained system by a process of selection of noiseless subsystems when the system is coupled to an external…
Complex systems are characterized by specific time-dependent interactions among their many constituents. As a consequence they often manifest rich, non-trivial and unexpected behavior. Examples arise both in the physical and non-physical…
Leveraging the compositional nature of our world to expedite learning and facilitate generalization is a hallmark of human perception. In machine learning, on the other hand, achieving compositional generalization has proven to be an…
Understanding the origins of complexity is a fundamental challenge with implications for biological and technological systems. Network theory emerges as a powerful tool to model complex systems. Networks are an intuitive framework to…
The concept of complexity appears in virtually all areas of knowledge. Its intuitive meaning shares similarities across fields, but disagreements between its details hinders a general definition, leading to a plethora of proposed…
Compositionality is a key strategy for addressing combinatorial complexity and the curse of dimensionality. Recent work has shown that compositional solutions can be learned and offer substantial gains across a variety of domains, including…
Though modern neural networks have achieved impressive performance in both vision and language tasks, we know little about the functions that they implement. One possibility is that neural networks implicitly break down complex tasks into…
Large monolithic generative models trained on massive amounts of data have become an increasingly dominant approach in AI research. In this paper, we argue that we should instead construct large generative systems by composing smaller…
Building complex software systems necessitates the use of component-based architectures. In theory, of the set of components needed for a design, only some small portion of them are "custom"; the rest are reused or refactored existing…
Let M be a filtered module. Some properties of elements of M are "generic" in the following sense: (being open/stable) if an element z of M has a property P then any approximation of z has P; (being dense) any element of M is approximated…
The science of complex networks is a new interdisciplinary branch of science which has arisen recently on the interface of physics, biology, social and computer sciences, and others. Its main goal is to discover general laws governing the…
We discuss in a statistical physics framework the idea that ``the whole is less than the parts'', as sometimes advocated by sociologists in view of the intrinsic complexity of humans, and try to reconcile this idea with the statistical…
A class of models intended to be as minimal and structureless as possible is introduced. Even in cases with simple rules, rich and complex behavior is found to emerge, and striking correspondences to some important core known features of…
Structural Causal Models (SCMs) provide a popular causal modeling framework. In this work, we show that SCMs are not flexible enough to give a complete causal representation of dynamical systems at equilibrium. Instead, we propose a…
Statistical laws describe regular patterns observed in diverse scientific domains, ranging from the magnitude of earthquakes (Gutenberg-Richter law) and metabolic rates in organisms (Kleiber's law), to the frequency distribution of words in…