English
Related papers

Related papers: Optimizing Token Usage on Large Language Model Con…

200 papers

Large Language Models (LLMs) are pivotal in advancing natural language processing but often struggle with complex reasoning tasks due to inefficient attention distributions. In this paper, we explore the effect of increased computed tokens…

Computation and Language · Computer Science 2024-06-25 Bingli Liao , Danilo Vasconcellos Vargas

Small Language Models (SLMs) have become increasingly important due to their efficiency and performance to perform various language tasks with minimal computational resources, making them ideal for various settings including on-device,…

The geometric evolution of token representations in large language models (LLMs) presents a fundamental paradox: while human language inherently organizes semantic information in low-dimensional spaces ($\sim 10^1$ dimensions), modern LLMs…

Computation and Language · Computer Science 2025-03-31 Zhuo-Yang Song , Zeyu Li , Qing-Hong Cao , Ming-xing Luo , Hua Xing Zhu

Recent advancements in large language models (LLMs) have catalyzed the rise of reasoning-intensive inference paradigms, where models perform explicit step-by-step reasoning before generating final answers. While such approaches improve…

Artificial Intelligence · Computer Science 2026-04-28 Zichuan Fu , Xian Wu , Guojing Li , Yejing Wang , Yijun Chen , Zihao Zhao , Yixuan Luo , Hanyu Yan , Yefeng Zheng , Xiangyu Zhao

Large language model (LLM) tokenizers act as structured compressors: by mapping text to discrete token sequences, they determine token count (and thus compute and context usage) and the statistical structure seen by downstream models.…

Information Theory · Computer Science 2026-01-15 Mete Erdogan , Abhiram Gorle , Shubham Chandak , Mert Pilanci , Tsachy Weissman

Large language models (LLMs) process and predict sequences containing text to answer questions, and address tasks including document summarization, providing recommendations, writing software and solving quantitative problems. We provide a…

Numerical Analysis · Mathematics 2026-02-02 Ricardo Baptista , Andrew Stuart , Son Tran

Future wireless networks are expected to incorporate diverse services that often lack general mathematical models. To address such black-box network management tasks, the large language model (LLM) optimizer framework, which leverages…

Information Theory · Computer Science 2025-07-04 Hoon Lee , Wentao Zhou , Merouane Debbah , Inkyu Lee

This research investigates the application of Large Language Models (LLMs) to augment conversational agents in process mining, aiming to tackle its inherent complexity and diverse skill requirements. While LLM advancements present novel…

Artificial Intelligence · Computer Science 2023-07-20 Urszula Jessen , Michal Sroka , Dirk Fahland

The language generation and reasoning capabilities of large language models (LLMs) have enabled conversational systems with impressive performance in a variety of tasks, from code generation, to composing essays, to passing STEM and legal…

Artificial Intelligence · Computer Science 2025-02-28 Konstantina Christakopoulou , Iris Qu , John Canny , Andrew Goodridge , Cj Adams , Minmin Chen , Maja Matarić

Small Language Models (SLMs) have gained substantial attention due to their ability to execute diverse language tasks successfully while using fewer computer resources. These models are particularly ideal for deployment in limited…

Computation and Language · Computer Science 2025-05-30 Tanjil Hasan Sakib , Md. Tanzib Hosain , Md. Kishor Morol

The rapid development of large language models (LLMs) has significantly transformed the field of artificial intelligence, demonstrating remarkable capabilities in natural language processing and moving towards multi-modal functionality.…

Large language models (LLMs) have shown promising efficacy across various tasks, becoming powerful tools in numerous aspects of human life. However, Transformer-based LLMs suffer a performance degradation when modeling long-term contexts…

Computation and Language · Computer Science 2026-03-23 Weiyao Luo , Suncong Zheng , Heming Xia , Weikang Wang , Yan Lei , Tianyu Liu , Shuang Chen , Zhifang Sui

Supply Chain Management requires addressing a variety of complex decision-making challenges, from sourcing strategies to planning and execution. Over the last few decades, advances in computation and information technologies have enabled…

Artificial Intelligence · Computer Science 2025-07-30 David Simchi-Levi , Konstantina Mellou , Ishai Menache , Jeevan Pathuri

While model serving has unlocked unprecedented capabilities, the high cost of serving large-scale models continues to be a significant barrier to widespread accessibility and rapid innovation. Compiler optimizations have long driven…

Machine Learning · Computer Science 2026-02-05 Annabelle Sujun Tang , Christopher Priebe , Rohan Mahapatra , Lianhui Qin , Hadi Esmaeilzadeh

Multimodal protein language models (PLMs) integrate sequence and token-based structural information, serving as a powerful foundation for protein modeling, generation, and design. However, the reliance on tokenizing 3D structures into…

Machine Learning · Computer Science 2025-06-13 Cheng-Yen Hsieh , Xinyou Wang , Daiheng Zhang , Dongyu Xue , Fei Ye , Shujian Huang , Zaixiang Zheng , Quanquan Gu

Large language models (LLMs) are in need of sufficient contexts to handle many critical applications, such as retrieval augmented generation and few-shot learning. However, due to the constrained window size, the LLMs can only access to the…

Computation and Language · Computer Science 2024-01-17 Ninglu Shao , Shitao Xiao , Zheng Liu , Peitian Zhang

The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains, reshaping the artificial general intelligence landscape. However, the increasing computational and memory demands of these models…

Computation and Language · Computer Science 2024-04-22 Tianyu Ding , Tianyi Chen , Haidong Zhu , Jiachen Jiang , Yiqi Zhong , Jinxin Zhou , Guangzhi Wang , Zhihui Zhu , Ilya Zharkov , Luming Liang

Optimization algorithms and large language models (LLMs) enhance decision-making in dynamic environments by integrating artificial intelligence with traditional techniques. LLMs, with extensive domain knowledge, facilitate intelligent…

Neural and Evolutionary Computing · Computer Science 2024-05-17 Sen Huang , Kaixiang Yang , Sheng Qi , Rui Wang

Classic AI planning problems have been revisited in the Large Language Model (LLM) era, with a focus of recent benchmarks on success rates rather than plan efficiency. We examine the degree to which frontier models reason optimally versus…

Artificial Intelligence · Computer Science 2026-04-06 Bernd Bohnet , Michael C. Mozer , Kevin Swersky , Wil Cunningham , Aaron Parisi , Kathleen Kenealy , Noah Fiedel

In task-oriented conversational AI evaluation, unsupervised methods poorly correlate with human judgments, and supervised approaches lack generalization. Recent advances in large language models (LLMs) show robust zeroshot and few-shot…

Computation and Language · Computer Science 2024-06-26 Jinghan Jia , Abi Komma , Timothy Leffel , Xujun Peng , Ajay Nagesh , Tamer Soliman , Aram Galstyan , Anoop Kumar