Related papers: Linear algebra with transformers
The modern data analyst must cope with data encoded in various forms, vectors, matrices, strings, graphs, or more. Consequently, statistical and machine learning models tailored to different data encodings are important. We focus on data…
We showcase applications of nonlinear algebra in the sciences and engineering. Our review is organized into eight themes: polynomial optimization, partial differential equations, algebraic statistics, integrable systems, configuration…
We introduce a new framework called linear algebraic number theory (LANT) that reformulates the number-theoretic problem as a regression model and solves it using matrix algebra. This framework restricts all computations to log space,…
Transformers have demonstrated remarkable success across various applications. However, the success of transformers have not been understood in theory. In this work, we give a case study of how transformers can be trained to learn a classic…
Solving a polynomial system, or computing an associated Gr\"obner basis, has been a fundamental task in computational algebra. However, it is also known for its notorious doubly exponential time complexity in the number of variables in the…
Probabilistic forecasting of high dimensional multivariate time series is a notoriously challenging task, both in terms of computational burden and distribution modeling. Most previous work either makes simple distribution assumptions or…
A deformation of the canonical algebra for kinematical observables of the quantum field theory in Minkowski space-time has been considered under the condition of Lorentz invariance. A relativistic invariant algebra obtained depends on…
Modern neural network architectures have shown remarkable success in several large-scale classification and prediction tasks. Part of the success of these architectures is their flexibility to transform the data from the raw input…
Natural linear and coalgebra transformations of tensor algebras are studied. The representations of certain combinatorial groups are given. These representations are connected to natural transformations of tensor algebras and to the groups…
The accurate solution of some of the main problems in numerical linear algebra (linear system solving, eigenvalue computation, singular value computation and the least squares problem) for a totally positive Bernstein-Vandermonde matrix is…
Large language models exhibit a puzzling inconsistency: they solve complex problems yet frequently fail on seemingly simpler ones. We investigate whether LLMs internally encode problem difficulty in a way that aligns with human judgment,…
Can the cross product be generalized? Why are the trace and determinant so important in matrix theory? What do all the coefficients of the characteristic polynomial represent? This paper describes a technique for `doodling' equations from…
Transformer-based large language models have achieved remarkable performance across various natural language processing tasks. However, they often struggle with seemingly easy tasks like arithmetic despite their vast capabilities. This…
The transformer-based pre-trained language models have been tremendously successful in most of the conventional NLP tasks. But they often struggle in those tasks where numerical understanding is required. Some possible reasons can be the…
Powerful generative artificial intelligence from large language models (LLMs) harnesses extensive computational resources for inference. In this work, we investigate the transformer architecture, a key component of these models, under the…
We study whether a Large Language Model can learn the deterministic sequence of trees generated by the iterated prime factorization of the natural numbers. Each integer is mapped into a rooted planar tree and the resulting sequence $…
In this book, there are five chapters: Systems of Linear Equations, Vector Spaces, Homogeneous Systems, Characteristic Equation of Matrix, and Matrix Dot Product. It has also exercises at the end of each chapter above to let students…
We investigate the mechanisms that arise when transformers are trained to solve arithmetic on sequences where tokens are variables whose meaning is determined only through their interactions in-context. While prior work has studied…
In this paper, we propose to consider various models of pattern recognition. At the same time, it is proposed to consider models in the form of two operators: a recognizing operator and a decision rule. Algebraic operations are introduced…
A significant effort has been made to train neural networks that replicate algorithmic reasoning, but they often fail to learn the abstract concepts underlying these algorithms. This is evidenced by their inability to generalize to data…