English

Speed and Conversational Large Language Models: Not All Is About Tokens per Second

Computation and Language 2025-02-25 v1 Artificial Intelligence
Authors: Javier Conde , Miguel González , Pedro Reviriego , Zhen Gao , Shanshan Liu , Fabrizio Lombardi
View on arXiv ↗ PDF ↗ DOI ↗

Abstract

The speed of open-weights large language models (LLMs) and its dependency on the task at hand, when run on GPUs, is studied to present a comparative analysis of the speed of the most popular open LLMs.

Keywords

large language model language modeling large language model inference

Cite

@article{arxiv.2502.16721,
  title  = {Speed and Conversational Large Language Models: Not All Is About Tokens per Second},
  author = {Javier Conde and Miguel González and Pedro Reviriego and Zhen Gao and Shanshan Liu and Fabrizio Lombardi},
  journal= {arXiv preprint arXiv:2502.16721},
  year   = {2025}
}

Related papers

View all related →
Hardware Architecture · Computer Science
A Survey on Hardware Accelerators for Large Language Models
Christoforos Kachris
2025-01-13
Hardware Architecture · Computer Science
Hardware Acceleration of LLMs: A comprehensive survey and comparison
Nikoletta Koilia, Christoforos Kachris
2024-09-06
Computation and Language · Computer Science
LLMs Get Lost In Multi-Turn Conversation
Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, Jennifer Neville
2025-05-12
Performance · Computer Science
A Comprehensive Performance Study of Large Language Models on Novel AI Accelerators
Murali Emani, Sam Foreman, Varuni Sastry, Zhen Xie +5
2023-10-10
Distributed, Parallel, and Cluster Computing · Computer Science
Scaling Performance of Large Language Model Pretraining
Alexander Interrante-Grant, Carla Varela-Rosa, Suhaas Narayan, Chris Connelly +1
2025-10-10
Machine Learning · Computer Science
A Quantitative Review on Language Model Efficiency Research
Meng Jiang, Hy Dang, Lingbo Tong
2023-06-06
Machine Learning · Computer Science
New Solutions on LLM Acceleration, Optimization, and Application
Yingbing Huang, Lily Jiaxin Wan, Hanchen Ye, Manvi Jha +4
2024-06-18
Computation and Language · Computer Science
Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
Weigao Sun, Jiaxi Hu, Yucheng Zhou, Jusen Du +11
2025-08-14
Computation and Language · Computer Science
Evaluating the Effectiveness of Cost-Efficient Large Language Models in Benchmark Biomedical Tasks
Israt Jahan, Md Tahmid Rahman Laskar, Chun Peng, Jimmy Huang
2025-07-21
Performance · Computer Science
A Review on Proprietary Accelerators for Large Language Models
Sihyeong Park, Jemin Lee, Byung-Soo Kim, Seokhun Jeon
2025-03-14
Computation and Language · Computer Science
Evaluating the Process Modeling Abilities of Large Language Models -- Preliminary Foundations and Results
Peter Fettke, Constantin Houy
2025-03-19
Computer Vision and Pattern Recognition · Computer Science
Efficient Multimodal Large Language Models: A Survey
Yizhang Jin, Jian Li, Yexin Liu, Tianjun Gu +9
2026-01-23
Computation and Language · Computer Science
Language-Specific Latent Process Hinders Cross-Lingual Performance
Zheng Wei Lim, Alham Fikri Aji, Trevor Cohn
2025-09-29
Performance · Computer Science
Deploying Open-Source Large Language Models: A performance Analysis
Yannis Bendi-Ouis, Dan Dutartre, Xavier Hinaut
2025-06-13
Machine Learning · Computer Science
Computational Bottlenecks of Training Small-scale Large Language Models
Saleh Ashkboos, Iman Mirzadeh, Keivan Alizadeh, Mohammad Hossein Sekhavat +3
2024-12-03
Performance · Computer Science
Dissecting the Runtime Performance of the Training, Fine-tuning, and Inference of Large Language Models
Longteng Zhang, Xiang Liu, Zeyu Li, Xinglin Pan +7
2023-12-04
Performance · Computer Science
LLMPerf: GPU Performance Modeling meets Large Language Models
Khoi N. M. Nguyen, Hoang Duy Nguyen Do, Huyen Thao Le, Thanh Tuan Dao
2025-03-17
Artificial Intelligence · Computer Science
A Survey on Collaborative Mechanisms Between Large and Small Language Models
Yi Chen, JiaHao Zhao, HaoHao Han
2025-05-13
Computation and Language · Computer Science
A Primer on Large Language Models and their Limitations
Sandra Johnson, David Hyland-Wood
2024-12-09
Computation and Language · Computer Science
The Efficiency Spectrum of Large Language Models: An Algorithmic Survey
Tianyu Ding, Tianyi Chen, Haidong Zhu, Jiachen Jiang +6
2024-04-22
Distributed, Parallel, and Cluster Computing · Computer Science
A Review on Edge Large Language Models: Design, Execution, and Applications
Yue Zheng, Yuhao Chen, Bin Qian, Xiufang Shi +2
2025-02-25
Software Engineering · Computer Science
Do AI Models Dream of Faster Code? An Empirical Study on LLM-Proposed Performance Improvements in Real-World Software
Lirong Yi, Gregory Gay, Philipp Leitner
2026-04-10
Computation and Language · Computer Science
Do Generative Large Language Models need billions of parameters?
Sia Gholami, Marwan Omar
2023-09-14
Hardware Architecture · Computer Science
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
Cong Guo, Feng Cheng, Zhixu Du, James Kiessling +20
2024-10-11
Distributed, Parallel, and Cluster Computing · Computer Science
Characterization of Large Language Model Development in the Datacenter
Qinghao Hu, Zhisheng Ye, Zerui Wang, Guoteng Wang +8
2024-04-05
R2 v1 2026-06-28T21:54:48.009Z