Related papers: Composing Linear Layers from Irreducibles
This is an introduction to geometric algebra, an alternative to traditional vector algebra that expands on it in two ways: 1. In addition to scalars and vectors, it defines new objects representing subspaces of any dimension. 2. It defines…
Efficient tensor computation is a cornerstone of modern deep learning (DL) workloads, yet existing approaches struggle to achieve flexible and performant design and implementation of tensor layouts -- mappings between logical tensors and…
In this text we develop the formalism of products and powers of linear codes under componentwise multiplication. As an expanded version of the author's talk at AGCT-14, focus is put mostly on basic properties and descriptive statements that…
An effective way to model the complex real world is to view the world as a composition of basic components of objects and transformations. Although humans through development understand the compositionality of the real world, it is…
Transformers are effective and efficient at modeling complex relationships and learning patterns from structured data in many applications. The main aim of this paper is to propose and design NLAFormer, which is a transformer-based…
Dimensionality reduction is an effective method for learning high-dimensional data, which can provide better understanding of decision boundaries in human-readable low-dimensional subspace. Linear methods, such as principal component…
Some of the simplest, yet most frequently used predictors in statistics and machine learning use weighted linear combinations of features. Such linear predictors can model non-linear relationships between features by adding interaction…
Although it is known that transformer language models (LMs) pass features from early layers to later layers, it is not well understood how this information is represented and routed by the model. We analyze a mechanism used in two LMs to…
We propose a novel approach to disentangle the generative factors of variation underlying a given set of observations. Our method builds upon the idea that the (unknown) low-dimensional manifold underlying the data space can be explicitly…
Classical nonlinear dimensionality reduction (NLDR) techniques like t-SNE, Isomap, and LLE excel at creating low-dimensional embeddings for data visualization but fundamentally lack the ability to map these embeddings back to the original…
We consider the decomposition of bounded linear operators on Hilbert spaces in terms of functions forming frames. Similar to the singular-value decomposition, the resulting frame decompositions encode information on the structure and…
We construct logarithmic conformal field theories starting from an ordinary conformal field theory -- with a chiral algebra C and the corresponding space of states V -- via a two-step construction: i) deforming the chiral algebra…
The screening of novel materials is an important topic in the field of materials science. Although traditional computational modeling, especially first-principles approaches, is a very useful and accurate tool to predict the properties of…
We present a deep convolutional decoder architecture that can generate volumetric 3D outputs in a compute- and memory-efficient manner by using an octree representation. The network learns to predict both the structure of the octree, and…
Transformers can generate predictions in two approaches: 1. auto-regressively by conditioning each sequence element on the previous ones, or 2. directly produce an output sequences in parallel. While research has mostly explored upon this…
Patterns, which are collections of elements arranged in regular or near-regular arrangements, are an important graphic art form and widely used due to their elegant simplicity and aesthetic appeal. When a pattern is encoded as a flat image…
The theme of the first two sections, is to prepare the framework of how from a ``complicated'' family of so called index models $I \in K_1$ we build many and/or complicated structures in a class $K_2$. The index models are…
Deep generative models like variational autoencoders approximate the intrinsic geometry of high dimensional data manifolds by learning low-dimensional latent-space variables and an embedding function. The geometric properties of these…
Building models capable of generating structured output is a key challenge for AI and robotics. While generative models have been explored on many types of data, little work has been done on synthesizing lidar scans, which play a key role…
In this paper, we make the case that Clifford algebra is the natural framework for root systems and reflection groups, as well as related groups such as the conformal and modular groups: The metric that exists on these spaces can always be…