Related papers: Linear algebra with transformers
In these six lectures, we examine what can be learnt about the behavior of multi-layer neural networks from the analysis of linear models. We first recall the correspondence between neural networks and linear models via the so-called lazy…
Constructing accurate and automatic solvers of math word problems has proven to be quite challenging. Prior attempts using machine learning have been trained on corpora specific to math word problems to produce arithmetic expressions in…
Modeling of multidimensional signal using tensor is more convincing than representing it as a collection of matrices. The tensor based approaches can explore the abundant spatial and temporal structures of the mutlidimensional signal. The…
Lie-theoretic structures of type $E_8$ (e.g., Lie groups and algebras, Hecke algebras and Kazhdan-Lusztig cells, ...) are considered to serve as a `gold standard' when it comes to judging the effectiveness of a general algorithm for solving…
The objective in this paper is to demonstrate that four of the most used techniques in applied mathematics, viz., Fourier series, Fourier transform, Laplace transform and the Fourier-Laplace transform can be introduced using eigenvalue…
The automorphisms of all 4-dimensional, real Lie Algebras are presented in a comprehensive way. Their action on the space of $4\times 4$, real, symmetric and positive definite, matrices, defines equivalence classes which are used for the…
Recently, with the significant developments in deep learning techniques, solving underdetermined inverse problems has become one of the major concerns in the medical imaging domain. Typical examples include undersampled magnetic resonance…
Learning good representations without supervision is still an open issue in machine learning, and is particularly challenging for speech signals, which are often characterized by long sequences with a complex hierarchical structure. Some…
We revisit the results on admissible transformations between normal linear systems of second-order ordinary differential equations with an arbitrary number of dependent variables under several appropriate gauges of the arbitrary elements…
Neural networks allow solving many ill-posed inverse problems with unprecedented performance. Physics informed approaches already progressively replace carefully hand-crafted reconstruction algorithms in real applications. However, these…
Linear probes and sparse autoencoders consistently recover meaningful structure from transformer representations -- yet why should such simple methods succeed in deep, nonlinear systems? We show this is not merely an empirical regularity…
The linear inverse problem is fundamental to the development of various scientific areas. Innumerable attempts have been carried out to solve different variants of the linear inverse problem in different applications. Nowadays, the rapid…
This is a survey article for "Handbook of Linear Algebra", 2nd ed., Chapman & Hall/CRC, 2014. An informal introduction to representations of quivers and finite dimensional algebras from a linear algebraist's point of view is given. The…
Meta-learning, or learning-to-learn, seeks to design algorithms that can utilize previous experience to rapidly learn new skills or adapt to new environments. Representation learning -- a key tool for performing meta-learning -- learns a…
Many problems in machine learning can be expressed by means of a graph with nodes representing training samples and edges representing the relationship between samples in terms of similarity, temporal proximity, or label information. Graphs…
Transformers are a type of neural network that have demonstrated remarkable performance across various domains, particularly in natural language processing tasks. Motivated by this success, research on the theoretical understanding of…
This is an exercise based approach to matrix groups. The idea is to collect a bunch of exercises at one place which anyone with basic knowledge of linear algebra can attempt to solve and learn matrix groups and algebraic groups.
There is strong interest in developing mathematical methods that can be used to understand complex neural networks used in image analysis. In this paper, we introduce techniques from Linear Algebra to model neural network layers as maps…
In this book we use only special types of intervals and introduce the notion of different types of interval linear algebras and interval vector spaces using the intervals of the form [0, a] where the intervals are from Zn or Z+ \cup {0} or…
In past few decades, tensor algebra also known as multi-linear algebra has been developed and customized as a tool to be used for various engineering applications. In particular, with the help of a special form of tensor contracted product,…