Related papers: When "Normalization Without Loss of Generality" Lo…
Strong student models can learn from weaker teachers: when trained on the predictions of a weaker model, a strong pretrained student can learn to correct the weak model's errors and generalize to examples where the teacher is not confident,…
Common practice in modern machine learning involves fitting a large number of parameters relative to the number of observations. These overparameterized models can exhibit surprising generalization behavior, e.g., ``double descent'' in the…
The compactness phenomenon is one of the featured aspects of structuralism in mathematics. In simple and broad words, a compactness property holds in a structure if a related property is satisfied by sufficiently many substructures of that…
An open scientific challenge is how to classify events with reliable measures of uncertainty, when we have a mechanistic model of the data-generating process but the distribution over both labels and latent nuisance parameters is different…
When are two germs of analytic systems conjugate or orbitally equivalent under an analytic change of coordinates in the neighborhood of a singular point? A way to answer is to use normal forms. But there are large classes of dynamical…
This dissertation studies a fundamental open challenge in deep learning theory: why do deep networks generalize well even while being overparameterized, unregularized and fitting the training data to zero error? In the first part of the…
We consider the untyped lambda calculus with constructors and recursively defined constants. We construct a domain-theoretic model such that any term not denoting bottom is strongly normalising provided all its `stratified approximations'…
A total set of states for which we have no resolution of the identity (a `pre-basis'), is considered in a finite dimensional Hilbert space. A dressing formalism renormalizes them into density matrices which resolve the identity, and makes…
Several machine learning models are defined for inputs of any size, such as graphs with different numbers of nodes and point clouds containing varying numbers of points. The universality properties of such any-dimensional models remain…
This paper offers a new perspective to ease the challenge of domain generalization, which involves maintaining robust results even in unseen environments. Our design focuses on the decision-making process in the final classifier layer.…
Counterfactual examples are one of the most commonly-cited methods for explaining the predictions of machine learning models in key areas such as finance and medical diagnosis. Counterfactuals are often discussed under the assumption that…
Model robustness indicates a model's capability to generalize well on unforeseen distributional shifts, including data corruptions and adversarial attacks. Data augmentation is one of the most prevalent and effective ways to enhance…
Simplicity is held by many to be the key to general intelligence. Simpler models tend to "generalise", identifying the cause or generator of data with greater sample efficiency. The implications of the correlation between simplicity and…
By construction, gauge theories require gauge fixing. In conventional approaches to spontaneously broken gauge theories, the choice of the Unitary ('t Hooft) gauge involves the sacrifice of manifest renormalizability (unitarity). It is…
Modular crossed product algebras have recently assumed an important role in perturbative quantum gravity as they lead to an intrinsic regularization of entanglement entropies by introducing quantum reference frames (QRFs) in place of…
A different general philosophy, to be called Full Randomness (FR), for the analysis of random effects models is presented, involving a notion of reducing or preferably eliminating fixed effects, at least formally. For example, under FR…
Valuation algebras abstract a large number of formalisms for automated reasoning and enable the definition of generic inference procedures. Many of these formalisms provide some notions of solutions. Typical examples are satisfying…
Regularization methods allow one to handle a variety of inferential problems where there are more covariates than cases. This allows one to consider a potentially enormous number of covariates for a problem. We exploit the power of these…
Weak-to-strong generalization, where a student model trained on imperfect labels generated by a weaker teacher nonetheless surpasses that teacher, has been widely observed but the mechanisms that enable it have remained poorly understood.…
Evaluation of counterfactual queries (e.g., "If A were true, would C have been true?") is important to fault diagnosis, planning, determination of liability, and policy analysis. We present a method of revaluating counterfactuals when the…