Related papers: Comments on the Du-Kakade-Wang-Yang Lower Bounds
We present and analyze a natural hierarchy of weak theories, develop analysis in them, and show that they are interpretable in bounded quantifier arithmetic $\text{I}\Delta_0$ (and hence in Robinson arithmetic Q). The strongest theories…
We study the problem of learning robust classifiers where the classifier will receive a perturbed input. Unlike robust PAC learning studied in prior work, here the clean data and its label are also adversarially chosen. We formulate this…
We present some extensions of Bernstein's concentration inequality for random matrices. This inequality has become a useful and powerful tool for many problems in statistics, signal processing and theoretical computer science. The main…
We study the difficulty of computing topological entropy of subshifts subjected to mixing restrictions. This problem is well-studied for multidimensional subshifts of finite type : there exists a threshold in the irreducibility rate where…
Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models. Fitting these models by maximizing the marginal likelihood or evidence is typically intractable, thus a common approximation…
Continual learning, or lifelong learning, is a formidable current challenge to machine learning. It requires the learner to solve a sequence of $k$ different learning tasks, one after the other, while retaining its aptitude for earlier…
We prove that the list replicability number of $d$-dimensional $\gamma$-margin half-spaces satisfies \[ \frac{d}{2}+1 \le \mathrm{LR}(H^d_\gamma) \le d, \] which grows with dimension. This resolves several open problems: $\bullet$ Every…
Consideration of some perturbatively calculated gauge-invariant expectation values of local noncomposite operators in pure Yang-Mills theory indicates that those expectation values which are not dimension specific, and which are well…
In binary classification, imbalance refers to situations in which one class is heavily under-represented. This issue is due to either a data collection process or because one class is indeed rare in a population. Imbalanced classification…
Contrastive learning is a highly successful technique for learning representations of data from labeled tuples, specifying the distance relations within the tuple. We study the sample complexity of contrastive learning, i.e. the minimum…
Adversarially robust PAC learning has proved to be challenging, with the currently best known learners [Montasser et al., 2021a] relying on improper methods based on intricate compression schemes, resulting in sample complexity exponential…
Learning disentangled representations of textual data is essential for many natural language tasks such as fair classification, style transfer and sentence generation, among others. The existent dominant approaches in the context of text…
An extended formulation of a polytope P is a polytope Q which can be projected onto P. Extended formulations of small size (i.e., number of facets) are of interest, as they allow to model corresponding optimization problems as linear…
We present and study approximate notions of dimensional and margin complexity, which correspond to the minimal dimension or norm of an embedding required to approximate, rather then exactly represent, a given hypothesis class. We show that…
Multi-index models - functions which only depend on the covariates through a non-linear transformation of their projection on a subspace - are a useful benchmark for investigating feature learning with neural nets. This paper examines the…
We develop new, easily computable exponential decay bounds for inverses of banded matrices, based on the quasiseparable representation of Green matrices. The bounds rely on a diagonal dominance hypothesis and do not require explicit…
Instance discriminative self-supervised representation learning has been attracted attention thanks to its unsupervised nature and informative feature representation for downstream tasks. In practice, it commonly uses a larger number of…
Intractable distributions present a common difficulty in inference within the probabilistic knowledge representation framework and variational methods have recently been popular in providing an approximate solution. In this article, we…
The study of arguments as abstract entities and their interaction as introduced by Dung (Artificial Intelligence 177, 1995) has become one of the most active research branches within Artificial Intelligence and Reasoning. A main issue for…
Recent research demonstrated that training large language models involves memorization of a significant fraction of training data. Such memorization can lead to privacy violations when training on sensitive user data and thus motivates the…