Related papers: Composing Linear Layers from Irreducibles
There has been recent interest in novel Clifford geometric invariants of linear transformations. This motivates the investigation of such invariants for a certain type of geometric transformation of interest in the context of root systems,…
Sparse autoencoders have become a standard tool for uncovering interpretable latent representations in neural networks. Yet salient concepts often span manifolds that current linear methods cannot capture without post hoc analysis. This…
Latent space geometry provides a rigorous and empirically valuable framework for interacting with the latent variables of deep generative models. This approach reinterprets Euclidean latent spaces as Riemannian through a pull-back metric,…
Clifford geometric algebras of multivectors are treated in detail. These algebras are build over a graded space and exhibit a grading or multivector structure. The careful study of the endomorphisms of this space makes it clear, that…
Deep neural networks are composed of layers of parametrised linear operations intertwined with non linear activations. In basic models, such as the multi-layer perceptron, a linear layer operates on a simple input vector embedding of the…
Classical Clifford theory studies the decomposition of simple $G$-modules into simple $H$-modules for some normal subgroup $H \triangleleft G$. In this paper we deal with chains of normal subgroups $1 \triangleleft G_1 \triangleleft \cdots…
In response to the development of recent efficient dense layers, this paper shows that something as simple as replacing linear components in pointwise convolutions with structured linear decompositions also produces substantial gains in the…
We reformulate minimalist grammars as partial functions on term algebras for strings and trees. Using filler/role bindings and tensor product representations, we construct homomorphisms for these data structures into geometric vector…
Clifford algebras are important structures in Geometric Algebra and Quantum Mechanics. They have allowed a formalization of the primitive operators in Quantum Theory. The algebras are built over vector spaces with dimension a power of 2…
Recollements of derived module categories are investigated, using a new technique, ladders of recollements, which are mutation sequences. The position in the ladder is shown to control whether a recollement restricts from unbounded to…
It can be shown that it is possible to find a representation of Hecke algebras within Clifford algebras of multivectors. These Clifford algebras possess a unique gradation and a possibly non-symmetric bilinear form. Hecke algebra…
Airborne LiDAR (Light Detection and Ranging) data is widely applied in building reconstruction, with studies reporting success in typical buildings. However, the reconstruction of curved buildings remains an open research problem. To this…
Sparse autoencoders are a standard tool for uncovering interpretable latent representations in neural networks. Yet, their interpretation depends on the inputs, making their isolated study incomplete. Polynomials offer a solution; they…
In this work, we leverage the linear algebraic structure of distributed word representations to automatically extend knowledge bases and allow a machine to learn new facts about the world. Our goal is to extract structured facts from…
Gated Linear Units (GLUs) have become a common building block in modern foundation models. Bilinear layers drop the non-linearity in the "gate" but still have comparable performance to other GLUs. An attractive quality of bilinear layers is…
Linear algebraic expressions are the essence of many computationally intensive problems, including scientific simulations and machine learning applications. However, translating high-level formulations of these expressions to efficient…
We investigate compositional structures in data embeddings from pre-trained vision-language models (VLMs). Traditionally, compositionality has been associated with algebraic operations on embeddings of words from a pre-existing vocabulary.…
Using a binary representation for basis elements of an algebra combined with a framework of multiplier and index functions, a connection has been established between the structure of a large class of algebras and the XOR componentwise…
Designing complex engineered systems requires managing tightly coupled trade-offs between subsystem capabilities and resource requirements. Monotone co-design provides a compositional language for such problems, but its generality does not…
Designers craft and edit graphic designs in a layer representation, but layer-based editing becomes impossible once composited into a raster image. In this work, we propose LayerD, a method to decompose raster graphic designs into layers…