English

Notes on the Mathematical Structure of GPT LLM Architectures

Machine Learning 2024-10-28 v1
Authors: Spencer Becker-Kahn
View on arXiv ↗ PDF ↗

Abstract

An exposition of the mathematics underpinning the neural network architecture of a GPT-3-style LLM.

Keywords

GPT large language model neural network architecture mathematical structure

Cite

@article{arxiv.2410.19370,
  title  = {Notes on the Mathematical Structure of GPT LLM Architectures},
  author = {Spencer Becker-Kahn},
  journal= {arXiv preprint arXiv:2410.19370},
  year   = {2024}
}

Comments

10 pages

Related papers

View all related →
Machine Learning · Computer Science
A Structural Approach to the Design of Domain Specific Neural Network Architectures
Gerrit Nolte
2023-01-24
Artificial Intelligence · Computer Science
Integrating Graphs with Large Language Models: Methods and Prospects
Shirui Pan, Yizhen Zheng, Yixin Liu
2023-11-14
Computation and Language · Computer Science
Graph Language Models
Moritz Plenz, Anette Frank
2024-06-04
Computation and Language · Computer Science
Evaluating Spatial Understanding of Large Language Models
Yutaro Yamada, Yihan Bao, Andrew K. Lampinen, Jungo Kasai +1
2024-04-16
Machine Learning · Computer Science
Mathematics of Deep Learning
Rene Vidal, Joan Bruna, Raja Giryes, Stefano Soatto
2017-12-14
Computers and Society · Computer Science
Cognitive network science reveals bias in GPT-3, ChatGPT, and GPT-4 mirroring math anxiety in high-school students
Katherine Abramski, Salvatore Citraro, Luigi Lombardi, Giulio Rossetti +1
2023-05-31
Physics and Society · Physics
A tutorial on networks in social systems: A mathematical modeling perspective
Heather Z. Brooks
2023-02-03
Machine Learning · Statistics
Multilayer Perceptron Algebra
Zhao Peng
2017-01-19
Computation and Language · Computer Science
Benchmarking GPT-4 on Algorithmic Problems: A Systematic Evaluation of Prompting Strategies
Flavio Petruzzellis, Alberto Testolin, Alessandro Sperduti
2024-07-12
Machine Learning · Computer Science
A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems
Alexandre Duval, Simon V. Mathis, Chaitanya K. Joshi, Victor Schmidt +6
2024-03-14
Machine Learning · Computer Science
Mathematics of Neural Networks (Lecture Notes Graduate Course)
Bart M. N. Smets
2024-03-11
Computation and Language · Computer Science
Mathematical Language Models: A Survey
Wentao Liu, Hanglei Hu, Jie Zhou, Yuyang Ding +7
2025-01-03
Computation and Language · Computer Science
Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad, Mehwish Fatima
2026-05-20
Neural and Evolutionary Computing · Computer Science
Neural network models
Plamen Dimitrov
2023-01-10
Hardware Architecture · Computer Science
Architectural Implications of Graph Neural Networks
Zhihui Zhang, Jingwen Leng, Lingxiao Ma, Youshan Miao +2
2021-12-28
Logic · Mathematics
Restructuring Logic
Joachim Derichs
2014-03-12
Machine Learning · Computer Science
Lecture Notes: Neural Network Architectures
Evelyn Herberg
2023-04-19
Machine Learning · Computer Science
Unraveling Arithmetic in Large Language Models: The Role of Algebraic Structures
Fu-Chieh Chang, You-Chen Lin, Pei-Yuan Wu
2025-04-22
Machine Learning · Computer Science
Machine-Learning Mathematical Structures
Yang-Hui He
2021-04-09
Artificial Intelligence · Computer Science
Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis
Chang Liu, Bo Wu
2023-09-12
Computation and Language · Computer Science
Large Language Models
Michael R. Douglas
2023-10-09
Machine Learning · Computer Science
Mathematical theory of deep learning
Philipp Petersen, Jakob Zech
2026-01-16
History and Overview · Mathematics
Linear Geometry: flats, ranks, regularity, parallelity
Taras Banakh, Ivan Hetman, Alex Ravsky, Vlad Pshyk
2026-04-08
Condensed Matter · Physics
The Architecture of Complex Systems
Vito Latora, Massimo Marchiori
2007-05-23
Computation and Language · Computer Science
A Survey on Neural Network Language Models
Kun Jing, Jungang Xu
2019-06-14