English
Related papers

Related papers: Inconsistent Tokenizations Cause Language Models t…

200 papers

Large Language Models (LLMs) are widely deployed in real-world applications, yet little is known about their training dynamics at the token level. Evaluation typically relies on aggregated training loss, measured at the batch level, which…

Computation and Language · Computer Science 2024-10-17 Andrea Pinto , Tomer Galanti , Randall Balestriero

Large language models (LLMs) have been found to produce hallucinations when the question exceeds their internal knowledge boundaries. A reliable model should have a clear perception of its knowledge boundaries, providing correct answers…

Computation and Language · Computer Science 2024-08-20 Shiyu Ni , Keping Bi , Lulu Yu , Jiafeng Guo

Natural Language Inference (NLI) tasks involving temporal inference remain challenging for pre-trained language models (LMs). Although various datasets have been created for this task, they primarily focus on English and do not address the…

Computation and Language · Computer Science 2023-06-21 Tomoki Sugimoto , Yasumasa Onoe , Hitomi Yanaka

This paper examines the reasoning capabilities of Large Language Models (LLMs) from a novel perspective, focusing on their ability to operate within formally specified, rule-governed environments. We evaluate four LLMs (Gemini 2.5 Pro and…

Artificial Intelligence · Computer Science 2026-02-24 Maciej Świechowski , Adam Żychowski , Jacek Mańdziuk

Large Language Models (LLMs) exhibit a puzzling disparity in their formal linguistic competence: while they learn some linguistic phenomena with near-perfect mastery, they often perform below chance on others, even after training on…

Computation and Language · Computer Science 2026-04-21 H S V N S Kowndinya Renduchintala , Sumit Bhatia

Language models typically tokenize raw text into sequences of subword identifiers from a predefined vocabulary, a process inherently sensitive to typographical errors, length variations, and largely oblivious to the internal structure of…

Computation and Language · Computer Science 2024-10-07 Yekun Chai , Yewei Fang , Qiwei Peng , Xuhong Li

In enhancing the fairness of Large Language Models (LLMs), evaluating social biases rooted in the cultural contexts of specific linguistic regions is essential. However, most existing Japanese benchmarks heavily rely on translating English…

Computation and Language · Computer Science 2026-04-02 Taihei Shiotani , Masahiro Kaneko , Naoaki Okazaki

Large-scale pretrained language models are the major driving force behind recent improvements in performance on the Winograd Schema Challenge, a widely employed test of common sense reasoning ability. We show, however, with a new diagnostic…

Computation and Language · Computer Science 2020-05-08 Mostafa Abdou , Vinit Ravishankar , Maria Barrett , Yonatan Belinkov , Desmond Elliott , Anders Søgaard

The widespread adoption of large language models (LLMs) makes it important to recognize their strengths and limitations. We argue that in order to develop a holistic understanding of these systems we need to consider the problem that they…

Computation and Language · Computer Science 2023-09-26 R. Thomas McCoy , Shunyu Yao , Dan Friedman , Matthew Hardy , Thomas L. Griffiths

In the last decade, the generalization and adaptation abilities of deep learning models were typically evaluated on fixed training and test distributions. Contrary to traditional deep learning, large language models (LLMs) are (i) even more…

Computation and Language · Computer Science 2024-10-17 Fırat Öncel , Matthias Bethge , Beyza Ermis , Mirco Ravanelli , Cem Subakan , Çağatay Yıldız

Public leaderboards increasingly suggest that large language models (LLMs) surpass human experts on benchmarks spanning academic knowledge, law, and programming. Yet most benchmarks are fully public, their questions widely mirrored across…

Artificial Intelligence · Computer Science 2026-03-18 Eshwar Reddy M , Sourav Karmakar

We introduce a comprehensive Linguistic Benchmark designed to evaluate the limitations of Large Language Models (LLMs) in domains such as logical reasoning, spatial intelligence, and linguistic understanding, among others. Through a series…

Artificial Intelligence · Computer Science 2024-06-04 Sean Williams , James Huckle

The pursuit of leaderboard rankings in Large Language Models (LLMs) has created a fundamental paradox: models excel at standardized tests while failing to demonstrate genuine language understanding and adaptability. Our systematic analysis…

Computation and Language · Computer Science 2024-12-06 Sourav Banerjee , Ayushi Agarwal , Eishkaran Singh

Large Language Models (LLMs) have demonstrated remarkable generalization across diverse tasks, leading individuals to increasingly use them as personal assistants and universal computing engines. Nevertheless, a notable obstacle emerges…

Machine Learning · Computer Science 2023-09-13 Dimitris Spathis , Fahim Kawsar

With the widespread application of Large Language Models (LLMs) to various domains, concerns regarding the trustworthiness of LLMs in safety-critical scenarios have been raised, due to their unpredictable tendency to hallucinate and…

Computation and Language · Computer Science 2024-11-04 Xin Qiu , Risto Miikkulainen

Large Language Models (LLMs) are increasingly used in tasks requiring interpretive and inferential accuracy. In this paper, we introduce ExpliCa, a new dataset for evaluating LLMs in explicit causal reasoning. ExpliCa uniquely integrates…

Computation and Language · Computer Science 2026-02-10 Martina Miliani , Serena Auriemma , Alessandro Bondielli , Emmanuele Chersoni , Lucia Passaro , Irene Sucameli , Alessandro Lenci

Ambiguous words or underspecified references require interlocutors to resolve them, often by relying on shared context and commonsense knowledge. Therefore, we systematically investigate whether Large Language Models (LLMs) can leverage…

Computation and Language · Computer Science 2025-09-22 Lukas Ellinger , Georg Groh

While pre-trained language models (LMs) have brought great improvements in many NLP tasks, there is increasing attention to explore capabilities of LMs and interpret their predictions. However, existing works usually focus only on a certain…

Computation and Language · Computer Science 2022-07-29 Yaozong Shen , Lijie Wang , Ying Chen , Xinyan Xiao , Jing Liu , Hua Wu

Large language model (LLM) judges have often been used alongside traditional, algorithm-based metrics for tasks like summarization because they better capture semantic information, are better at reasoning, and are more robust to…

Computation and Language · Computer Science 2026-02-10 Jiangnan Fang , Cheng-Tse Liu , Hanieh Deilamsalehy , Nesreen K. Ahmed , Puneet Mathur , Nedim Lipka , Franck Dernoncourt , Ryan A. Rossi

Large language models (LLMs) have demonstrated remarkable performances on a wide range of natural language tasks. Yet, LLMs' successes have been largely restricted to tasks concerning words, sentences, or documents, and it remains…

Computation and Language · Computer Science 2024-07-24 Andrew Shin , Kunitake Kaneko