Related papers: Labeled compression schemes for extremal classes
We say that a set system $\mathcal{F}\subseteq 2^{[n]}$ shatters a given set $S\subseteq [n]$ if $2^S={F \cap S : F \in \mathcal{F}}$. The Sauer inequality states that in general, a set system $\mathcal{F}$ shatters at least $|\mathcal{F}|$…
Recently, a series of works have started studying variations of concepts from learning theory for product spaces, which can be collected under the name high-arity learning theory. In this work, we consider a high-arity variant of sample…
For any family of measurable sets in a probability space, we show that either (i) the family has infinite Vapnik-Chervonenkis (VC) dimension or (ii) for every epsilon > 0 there is a finite partition pi such the pi-boundary of each set has…
We begin this report by describing the Probably Approximately Correct (PAC) model for learning a concept class, consisting of subsets of a domain, and a function class, consisting of functions from the domain to the unit interval. Two…
Well-graded families, extremal systems and maximum systems (the last two in the sense of VC-theory and Sauer-Shelah lemma on VC-dimension) are three important classes of set systems. This paper aims to study the notion of duality in the…
Support Vector Machines (SVMs) are one of the most popular supervised learning models to classify using a hyperplane in an Euclidean space. Similar to SVMs, tropical SVMs classify data points using a tropical hyperplane under the tropical…
What are called secondary characteristic classes in Chern-Weil theory are a refinement of ordinary characteristic classes of principal bundles from cohomology to differential cohomology. We consider the problem of refining the construction…
Constant weight codes (CWCs) and constant composition codes (CCCs) are two important classes of codes that have been studied extensively in both combinatorics and coding theory for nearly sixty years. In this paper we show that for {\it…
One of the open problems in machine learning is whether any set-family of VC-dimension $d$ admits a sample compression scheme of size $O(d)$. In this paper, we study this problem for balls in graphs. For a ball $B=B_r(x)$ of a graph…
We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the…
Extreme multi-label classification or XMLC, is an active area of interest in machine learning. Compared to traditional multi-label classification, here the number of labels is extremely large, hence, the name extreme multi-label…
We say that a set system $\mathcal{F}\subseteq 2^{[n]}$ shatters a given set $S\subseteq [n]$ if $2^S=\{F \cap S : F \in \mathcal{F}\}$. The Sauer inequality states that in general, a set system $\mathcal{F}$ shatters at least…
In the affine fragment of continuous logic, type spaces are compact convex sets. I study some model theoretic properties of extreme types. It is proved that every complete theory $T$ has an extremal model, i.e. a model which realizes only…
In the affine space $\mathbb{F}_q^n$ over the finite field of order $q$, a point set $S$ is said to be $(d,k,r)$-evasive if the intersection between $S$ and any variety, of dimension $k$ and degree at most $d$, has cardinality less than…
We study the problem of compression for the purpose of similarity identification, where similarity is measured by the mean square Euclidean distance between vectors. While the asymptotical fundamental limits of the problem - the minimal…
Vapnik-Chervonenkis (VC) theory has so far been unable to explain the small generalization error of overparametrized neural networks. Indeed, existing applications of VC theory to large networks obtain upper bounds on VC dimension that are…
We analyze a family of supervised learning algorithms based on sample compression schemes that are stable, in the sense that removing points from the training set which were not selected for the compression set does not alter the resulting…
The aim of this paper is to provide several novel upper bounds on the excess risk with a primal focus on classification problems. We suggest two approaches and the obtained bounds are represented via the distribution dependent local…
Let $X$ be a finite set in a complex sphere of $d$ dimension. Let $D(X)$ be the set of usual inner products of two distinct vectors in $X$. A set $X$ is called a complex spherical $s$-code if the cardinality of $D(X)$ is $s$ and $D(X)$…
VC-dimension and VC-density are measures of combinatorial complexity of set systems. VC-dimension was first introduced in the context of statistical learning theory, and is tightly related to the sample complexity in PAC learning.…