Related papers: Generalized Householder transformations
To process novel sentences, language models (LMs) must generalize compositionally -- combine familiar elements in new ways. What aspects of a model's structure promote compositional generalization? Focusing on transformers, we test the…
Using a key observation due to Thiemann, a generalized Wick transform is introduced to map the constraint functionals of Riemannian general relativity to those of the Lorentzian theory, including matter sources. This opens up a new avenue…
The formalism which has been developed to give general expressions for the determinants of differential operators is extended to the physically interesting situation where these operators have a zero mode which has been extracted. In the…
W-transforms are introduced as uniformity-preserving univariate transformations on the unit interval induced by distribution functions and piecewise strictly monotone functions, and their properties are investigated. When applied…
We propose a probabilistic interpretation of transformers as unrolled inference steps assuming a probabilistic Laplacian Eigenmaps model from the ProbDR framework. Our derivation shows that at initialisation, transformers perform "linear"…
Exploiting symmetry in dynamical systems is a powerful way to improve the generalization of deep learning. The model learns to be invariant to transformation and hence is more robust to distribution shift. Data augmentation and equivariant…
When writing programs, people have the ability to tackle a new complex task by decomposing it into smaller and more familiar subtasks. While it is difficult to measure whether neural program synthesis methods have similar capabilities, what…
There have been many recent advances in the structure and measurement of distributed language models: those that map from words to a vector-space that is rich in information about word choice and composition. This vector-space is the…
Deep neural networks are usually trained in the space of the nodes, by adjusting the weights of existing links via suitable optimization protocols. We here propose a radically new approach which anchors the learning process to reciprocal…
Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize…
We propose Differentiable Window, a new neural module and general purpose component for dynamic window selection. While universally applicable, we demonstrate a compelling use case of utilizing Differentiable Window to improve standard…
Generalization of models to out-of-distribution (OOD) data has captured tremendous attention recently. Specifically, compositional generalization, i.e., whether a model generalizes to new structures built of components observed during…
Compositional generalization is one of the main properties which differentiates lexical learning in humans from state-of-art neural networks. We propose a general framework for building models that can generalize compositionally using the…
Partial transpose is an important operation for quantifying the entanglement, here we study the (partial) transpose of any single (two-mode) operators. Using the Fock-basis expansion, it is found that the transposed operator of an arbitrary…
Generalization methods offer a powerful solution to one of the key drawbacks of randomized controlled trials (RCTs): their limited representativeness. By enabling the transport of treatment effect estimates to target populations subject to…
We present a generic programming framework for OCAML which makes it possible to implement extensible transformations for a large scale of type definitions. Our framework makes use of objectoriented features of OCAML, utilising late binding…
Quantum general relativity may be considered as generally covariant QFT on differentiable manifolds, without any a priori metric structure. The kinematically covariance group acts by general diffeomorphisms on the manifold and by…
A generalization of a distribution increases the flexibility particularly in studying of a phenomenon and its properties. Many generalizations of continuous univariate distributions are available in literature. In this study, an…
Group equivariant neural networks are used as building blocks of group invariant neural networks, which have been shown to improve generalisation performance and data efficiency through principled parameter sharing. Such works have mostly…
The notion of Fourier transform is among the more important tools in analysis, which has been generalized in abstract harmonic analysis to the level of abelian locally compact groups. The aim of this paper is to further generalize the…