Related papers: Composing Linear Layers from Irreducibles
Relational machine learning programs like those developed in Inductive Logic Programming (ILP) offer several advantages: (1) The ability to model complex relationships amongst data instances; (2) The use of domain-specific relations during…
In this paper, we propose to exploit the rich hierarchical features of deep convolutional neural networks to improve the accuracy and robustness of visual tracking. Deep neural networks trained on object recognition datasets consist of…
A method is given that "inverts" a logic grammar and displays it from the point of view of the logical form, rather than from that of the word string. LR-compiling techniques are used to allow a recursive-descent generation algorithm to…
A recent body of work has demonstrated that Transformer embeddings can be linearly decomposed into well-defined sums of factors, that can in turn be related to specific network inputs or components. There is however still a dearth of work…
Logic-based Benders decomposition (LBBD) is a substantial generalization of classical Benders decomposition that, in principle, allows the subproblem to be any optimization problem rather than specifically a linear or nonlinear programming…
Geometric algebra is a powerful framework that unifies mathematics and physics. Since its revival in the middle of the 1960s by David Hestenes, it attracts great attention and has been exploited in many fields such as physics, computer…
Due to the substantial scale of Large Language Models (LLMs), the direct application of conventional compression methodologies proves impractical. The computational demands associated with even minimal gradient updates present challenges,…
In this Master of Science Thesis I introduce geometric algebra both from the traditional geometric setting of vector spaces, and also from a more combinatorial view which simplifies common relations and operations. This view enables us to…
Decoder-only transformers have become the standard architecture for large language models (LLMs) due to their strong performance. Recent studies suggest that, in pre-trained LLMs, early, middle, and late layers may serve distinct roles:…
We develop a wide general theory of bilinear bi-parameter singular integrals $T$. First, we prove a dyadic representation theorem starting from $T1$ assumptions and apply it to show many estimates, including $L^p \times L^q \to L^r$…
In this paper, we develop a representation-theoretic formulation of discrete-time linear systems. We show that such systems are naturally viewed as representations of time groups acting on vector spaces, thereby endowing the state space…
Complex Lie point transformations are used to linearize a class of systems of second order ordinary differential equations (ODEs) which have Lie algebras of maximum dimension $d$, with $d\leq 4$. We identify such a class by employing…
Nonexpansive mappings play a central role in modern optimization and monotone operator theory because their fixed points can describe solutions to optimization or critical point problems. It is known that when the mappings are sufficiently…
Classical and exceptional Lie algebras and their representations are among the most important tools in the analysis of symmetry in physical systems. In this letter we show how the computation of tensor products and branching rules of…
The geometric calculus based on Clifford algebra is a very useful tool for geometry and physics. It describes a geometric structure which is much richer than the ordinary geometry of spacetime. A Clifford manifold (C-space) consists not…
Model reduction of high-dimensional dynamical systems alleviates computational burdens faced in various tasks from design optimization to model predictive control. One popular model reduction approach is based on projecting the governing…
Principal angles are used to define an angle bivector of subspaces, which fully describes their relative inclination. Its exponential is related to the Clifford geometric product of blades, gives rotors connecting subspaces via minimal…
While the existence of low-dimensional embedding manifolds has been shown in patterns of collective motion, the current battery of nonlinear dimensionality reduction methods are not amenable to the analysis of such manifolds. This is mainly…
Many areas of machine learning and science involve large linear algebra problems, such as eigendecompositions, solving linear systems, computing matrix exponentials, and trace estimation. The matrices involved often have Kronecker,…
Human language understanding operates at multiple levels of granularity (e.g., words, phrases, and sentences) with increasing levels of abstraction that can be hierarchically combined. However, existing deep models with stacked layers do…