Related papers: Density of compressible types and some consequence…
We study invariant types in NIP theories. Amongst other things: we prove a definable version of the (p,q)-theorem in theories of small or medium directionality; we construct a canonical retraction from the space of M-invariant types to that…
In this paper, we explore connections between interpretable machine learning and learning theory through the lens of local approximation explanations. First, we tackle the traditional problem of performance generalization and bound the…
Combining two results from machine learning theory we prove that a formula is NIP if and only if it satisfies uniform definability of types over finite sets (UDTFS). This settles a conjecture of Laskowski.
We study neural network compressibility by using singular learning theory to extend the minimum description length (MDL) principle to singular models like neural networks. Through extensive experiments on the Pythia suite with quantization,…
We continue investigating the structure of externally definable sets in NIP theories and preservation of NIP after expanding by new predicates. Most importantly: types over finite sets are uniformly definable; over a model, a family of…
We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability…
We consider the problem of estimating the density $\Pi$ of a determinantal process $N$ from the observation of $n$ independent copies of it. We use an aggregation procedure based on robust testing to build our estimator. We establish…
The following strong form of density of definable types is introduced for theories T admitting a fibered dimension function d: given a model M of T and a definable subset X of M^n, there is a definable type p in X, definable over a code for…
Deep neural networks generalize well on unseen data though the number of parameters often far exceeds the number of training examples. Recently proposed complexity measures have provided insights to understanding the generalizability in…
The law of large numbers for the empirical density for the pairs of uniformly distributed integers with a given greatest common divisor is a classic result in number theory. In this paper, we study the large deviations of the empirical…
The increasing deployment of machine learning as well as legal regulations such as EU's GDPR cause a need for user-friendly explanations of decisions proposed by machine learning models. Counterfactual explanations are considered as one of…
This paper is a modified chapter of the author's Ph.D. thesis. We introduce the notions of sequentially approximated types and sequentially approximated Keisler measures. As the names imply, these are types which can be approximated by a…
Introducing the notion of a rational system of measure preserving transformations and proving a recurrence result for such systems, we give sufficient conditions in order a subset of rational numbers to contain arbitrary long arithmetic…
We use a formal correspondence between thermodynamics and inference, where the number of samples can be thought of as the inverse temperature, to study a quantity called ``learning capacity'' which is a measure of the effective…
This paper focuses on the relation between computational learning theory and resource-bounded dimension. We intend to establish close connections between the learnability/nonlearnability of a concept class and its corresponding size in…
We investigate the teaching of infinite concept classes through the effect of the learning bias (which is used by the learner to prefer some concepts over others and by the teacher to devise the teaching examples) and the sampling bias…
We examine the convergence properties of sequences of nonnegative real numbers that satisfy a particular class of recursive inequalities, from the perspective of proof theory and computability theory. We first establish a number of results…
A brief survey of the author and collaborators' work in compressive sensing applications to continuous imaging models.
Learning Spaces are certain set systems that are applied in the mathematical modeling of education. We propose a suitable compression (without loss of information) of such set systems to facilitate their logical and statistical analysis.…
We propose a neural hybrid model consisting of a linear model defined on a set of features computed by a deep, invertible transformation (i.e. a normalizing flow). An attractive property of our model is that both p(features), the density of…