Related papers: Is Complexity an Illusion?
Obtaining compositional mappings is important for the model to generalize well compositionally. To better understand when and how to encourage the model to learn such mappings, we study their uniqueness through different perspectives.…
Selective rationalization has become a common mechanism to ensure that predictive models reveal how they use any available features. The selection may be soft or hard, and identifies a subset of input features relevant for prediction. The…
One of the main goals of scientific research is to provide a description of the empirical data which is as accurate and comprehensive as possible, while relying on as few and simple assumptions as possible. In this paper, I propose a…
Objects are a centerpiece of the mathematical realm and our interaction with and reasoning about it, just as they are of the physical one (if not more). And humans' mathematical reasoning must ultimately be grounded in our general…
Usually, density functional models are considered approximations to density functional theory, However, there is no systematic connection between the two, and this can make us doubt about a linkage. This attitude can be further enforced by…
The standard assumptions that underlie many conceptual and quantitative frameworks do not hold for many complex physical, biological, and social systems. Complex systems science clarifies when and why such assumptions fail and provides…
Machine learning models that are developed with invariance to certain types of data transformations have demonstrated superior generalization performance in practice. However, the underlying mechanism that explains why invariance leads to…
Globalised-civilisation interaction intensity grows exponentially, involving all dimensions and regions of planetary environment. The resulting dynamics of critically high, exploding complexity urgently needs consistent understanding and…
The goal of machine learning is to find models that minimize prediction error on data that has not yet been seen. Its operational paradigm assumes access to a dataset $S$ and articulates a scheme for evaluating how well a given model…
In designing an intelligent system that must be able to explain its reasoning to a human user, or to provide generalizations that the human user finds reasonable, it may be useful to take into consideration psychological data on what types…
Compositional generalization is the capacity to recognize and imagine a large amount of novel combinations from known components. It is a key in human intelligence, but current neural networks generally lack such ability. This report…
We propose a generalization of small world networks, in which the reconnection of links is governed by a function that depends on the distance between the elements to be linked. An adequate choice of this function lets us control the…
The theory of computational complexity focuses on functions and, hence, studies programs whose interactive behavior is reduced to a simple question/answer pattern. We propose a broader theory whose ultimate goal is expressing and analyzing…
Natural language is compositional; the meaning of a sentence is a function of the meaning of its parts. This property allows humans to create and interpret novel sentences, generalizing robustly outside their prior experience. Neural…
We derive generalization bounds for learning algorithms based on their robustness: the property that if a testing sample is "similar" to a training sample, then the testing error is close to the training error. This provides a novel…
Language provides simple ways of communicating generalizable knowledge to each other (e.g., "Birds fly", "John hikes", "Fire makes smoke"). Though found in every language and emerging early in development, the language of generalization is…
Why do collectives outperform individuals when solving some problems? Fundamentally, collectives have greater computational resources with more sensory information, more memory, more processing capacity, and more ways to act. While greater…
We consider concept generalization at a large scale in the diverse and natural visual spectrum. Established computational modes (i.e., rule-based or similarity-based) are primarily studied isolated and focus on confined and abstract problem…
How do neural network image classifiers respond to simpler and simpler inputs? And what do such responses reveal about the learning process? To answer these questions, we need a clear measure of input simplicity (or inversely, complexity),…
The generalization gap of a classifier is related to the complexity of the set of functions among which the classifier is chosen. We study a family of low-complexity classifiers consisting of thresholding a random one-dimensional feature.…