Related papers: Statistical Learning Theory of Quasi-Regular Cases
Sufficiency, Conditionality and Invariance are basic principles of statistical inference. Current mathematical statistics courses do not devote much teaching time to these classical principles, and even ignore the latter two, in order to…
The classical theory of rank-based inference is entirely based either on ordinary ranks, which do not allow for considering location (intercept) parameters, or on signed ranks, which require an assumption of symmetry. If the median, in the…
In statistical inference, uncertainty is unknown and all models are wrong. That is to say, a person who makes a statistical model and a prior distribution is simultaneously aware that both are fictional candidates. To study such cases,…
This paper reviews the most common situations where one or more regularity conditions which underlie classical likelihood-based parametric inference fail. We identify three main classes of problems: boundary problems, indeterminate…
We study non-Bayesian social learning on random directed graphs and show that under mild connectivity assumptions, all the agents almost surely learn the true state of the world asymptotically in time if the sequence of the associated…
In scientific machine learning, models are routinely deployed with parameter values or boundary conditions far from those used in training. This paper studies the learning-where-to-learn problem of designing a training data distribution…
We introduce a new notion of generalization -- Distributional Generalization -- which roughly states that outputs of a classifier at train and test time are close *as distributions*, as opposed to close in just their average error. For…
Many machine learning models appear to deploy effortlessly under distribution shift, and perform well on a target distribution that is considerably different from the training distribution. Yet, learning theory of distribution shift bounds…
We investigate joint spectral characteristics of a family of matrices $\mathcal F $, associated with products in the semigroup generated by $\mathcal F$. In the literature, extremal measures such as the well-known joint spectral radius and…
Assumptions about invariances or symmetries in data can significantly increase the predictive power of statistical models. Many commonly used models in machine learning are constraint to respect certain symmetries in the data, such as…
We prove an impossibility result, which in the context of function learning says the following: under certain conditions, it is impossible to simultaneously learn symmetries and functions equivariant under them using an ansatz consisting of…
Generative models at times produce "invalid" outputs, such as images with generation artifacts and unnatural sounds. Validity-constrained distribution learning attempts to address this problem by requiring that the learned distribution have…
Quasi-logarithmic combinatorial structures are a class of decomposable combinatorial structures which extend the logarithmic class considered by Arratia, Barbour and Tavar\'{e} (2003). In order to obtain asymptotic approximations to their…
In some estimation problems, especially in applications dealing with information theory, signal processing and biology, theory provides us with additional information allowing us to restrict the parameter space to a finite number of points.…
The main goal of statistical learning theory is to provide a fundamental framework for the problem of decision making and model construction based on sets of data. Here, we present a brief introduction to the fundamentals of statistical…
We study statistical properties of the random variables $X_{\sigma}(\pi)$, the number of occurrences of the pattern $\sigma$ in the permutation $\pi$. We present two contrasting approaches to this problem: traditional probability theory and…
Negativities in quasiprobability distributions, a foundational concept originating in quantum optics, serve as a fundamental signature of quantum nonclassicality, with entanglement quasiprobabilities offering a necessary and sufficient…
When the historical data are limited, the conditional probabilities associated with the nodes of Bayesian networks are uncertain and can be empirically estimated. Second order estimation methods provide a framework for both estimating the…
Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity. An alternative approach, mainly studied in the statistical physics literature, is…
Early in training, LMs can behave like n-gram models, but eventually they often learn tree-based syntactic rules and generalize hierarchically out of distribution (OOD). We study this shift using controlled grammar-learning tasks: question…