Related papers: Learning to be Simple
We study from a statistical mechanics viewpoint some of the simplest mathematical objects, finite pure sets. Starting from the empty set, new generations are produced step by step, sets of the next generation being those whose elements are…
It is well understood that the structure of a social network is critical to whether or not agents can aggregate information correctly. In this paper, we study social networks that support information aggregation when rational agents act…
We consider whether given a simple, finite description of a group in the form of an algorithm, it is possible to algorithmically determine if the corresponding group has some specified property or not. When there is such an algorithm, we…
Probabilistic graphical models combine the graph theory and probability theory to give a multivariate statistical modeling. They provide a unified description of uncertainty using probability and complexity using the graphical model.…
Several works have proposed Simplicity Bias (SB)---the tendency of standard training procedures such as Stochastic Gradient Descent (SGD) to find simple models---to justify why neural networks generalize well [Arpit et al. 2017, Nakkiran et…
Building explainable systems is a critical problem in the field of Natural Language Processing (NLP), since most machine learning models provide no explanations for the predictions. Existing approaches for explainable machine learning…
In this thesis, we develop various techniques for working with sets in machine learning. Each input or output is not an image or a sequence, but a set: an unordered collection of multiple objects, each object described by a feature vector.…
How do neural networks trained over sequences acquire the ability to perform structured operations, such as arithmetic, geometric, and algorithmic computation? To gain insight into this question, we introduce the sequential group…
Simplicity bias, the propensity of deep models to over-rely on simple features, has been identified as a potential reason for limited out-of-distribution generalization of neural networks (Shah et al., 2020). Despite the important…
We present convincing empirical evidence for an effective and general strategy for building accurate small models. Such models are attractive for interpretability and also find use in resource-constrained environments. The strategy is to…
Asymptotic properties of finitely generated subgroups of free groups, and of finite group presentations, can be considered in several fashions, depending on the way these objects are represented and on the distribution assumed on these…
In this paper we prove a series of matching theorems for two sets of Coxeter generators of a finitely generated Coxeter group that identify common features of the two sets of generators. As an application, we describe an algorithm for…
We explore the possibility of using machine learning to identify interesting mathematical structures by using certain quantities that serve as fingerprints. In particular, we extract features from integer sequences using two empirical laws:…
Neural networks often exhibit simplicity bias, favoring simpler features over more complex ones, even when both are equally predictive. We introduce a novel method called imbalanced label coupling to explore and extend this simplicity bias…
Neural networks trained with SGD were recently shown to rely preferentially on linearly-predictive features and can ignore complex, equally-predictive ones. This simplicity bias can explain their lack of robustness out of distribution…
Humans can systematically generalize to novel compositions of existing concepts. Recent studies argue that neural networks appear inherently ineffective in such cognitive capacity, leading to a pessimistic view and a lack of attention to…
The existence of an infinite simple boundedly generated 2-generated group and the existence of a boundedly simple 2-generated group containing a free non-cyclic subgroup are proved.
Machine learning is applied to find proofs, with smaller or smallest numbers of nodes, for the classification of 4-nilpotent semigroups.
Dixon's famous theorem states that the group generated by two random permutations of a finite set is generically either the whole symmetric group or the alternating group. In the context of random generation of finite groups this means that…
We consider the problem of distribution-free conformal prediction and the criterion of group conditional validity. This criterion is motivated by many practical scenarios including hidden stratification and group fairness. Existing methods…