English
Related papers

Related papers: To token or not to token: A Comparative Study of T…

200 papers

Large language models have demonstrated robust performance on various language tasks using zero-shot or few-shot learning paradigms. While being actively researched, multimodal models that can additionally handle images as input have yet to…

Computation and Language · Computer Science 2023-05-24 Sherzod Hakimov , David Schlangen

Recent work on speech representation models jointly pre-trained with text has demonstrated the potential of improving speech representations by encoding speech and text in a shared space. In this paper, we leverage such shared…

Computation and Language · Computer Science 2023-10-10 Chung-Ming Chien , Mingjiamei Zhang , Ju-Chieh Chou , Karen Livescu

Pre-trained language models are still far from human performance in tasks that need understanding of properties (e.g. appearance, measurable quantity) and affordances of everyday objects in the real world since the text lacks such…

Computation and Language · Computer Science 2022-03-18 Woojeong Jin , Dong-Ho Lee , Chenguang Zhu , Jay Pujara , Xiang Ren

This paper investigates how Large Language Models (LLMs) represent non-English tokens -- a question that remains underexplored despite recent progress. We propose a lightweight intervention method using representation steering, where a…

Computation and Language · Computer Science 2025-08-27 Omar Mahmoud , Buddhika Laknath Semage , Thommen George Karimpanal , Santu Rana

Multilingual BERT (mBERT), a language model pre-trained on large multilingual corpora, has impressive zero-shot cross-lingual transfer capabilities and performs surprisingly well on zero-shot POS tagging and Named Entity Recognition (NER),…

Computation and Language · Computer Science 2022-05-18 Beiduo Chen , Wu Guo , Quan Liu , Kun Tao

While cross-linguistic model transfer is effective in many settings, there is still limited understanding of the conditions under which it works. In this paper, we focus on assessing the role of lexical semantics in cross-lingual transfer,…

Computation and Language · Computer Science 2024-08-15 Roy Ilani , Taelin Karidi , Omri Abend

Many natural language processing (NLP) tasks involve reasoning with textual spans, including question answering, entity recognition, and coreference resolution. While extensive research has focused on functional architectures for…

Computation and Language · Computer Science 2020-06-09 Shubham Toshniwal , Haoyue Shi , Bowen Shi , Lingyu Gao , Karen Livescu , Kevin Gimpel

Graph models are fundamental to data analysis in domains rich with complex relationships. Text-to-Graph-Query-Language (Text-to-GQL) systems act as a translator, converting natural language into executable graph queries. This capability…

Artificial Intelligence · Computer Science 2026-02-13 Songlin Lyu , Lujie Ban , Zihang Wu , Tianqi Luo , Jirong Liu , Chenhao Ma , Yuyu Luo , Nan Tang , Shipeng Qi , Heng Lin , Yongchao Liu , Chuntao Hong

Text classification approaches have usually required task-specific model architectures and huge labeled datasets. Recently, thanks to the rise of text-based transfer learning techniques, it is possible to pre-train a language model in an…

Computation and Language · Computer Science 2019-06-10 Enkhbold Bataa , Joshua Wu

Large language models have demonstrated predictable scaling behaviors with respect to model parameters and training data. This study investigates whether a similar scaling relationship exist for vision-language models with respect to the…

Artificial Intelligence · Computer Science 2025-12-30 Tenghui Li , Guoxu Zhou , Xuyang Zhao , Qibin Zhao

Without any explicit cross-lingual training data, multilingual language models can achieve cross-lingual transfer. One common way to improve this transfer is to perform realignment steps before fine-tuning, i.e., to train the model to build…

Computation and Language · Computer Science 2023-06-06 Félix Gaschi , Patricio Cerda , Parisa Rastin , Yannick Toussaint

Tokenization is a necessary component within the current architecture of many language mod-els, including the transformer-based large language models (LLMs) of Generative AI, yet its impact on the model's cognition is often overlooked. We…

Character-level language models obviate the need for separately trained tokenizers, but efficiency suffers from longer sequence lengths. Learning to combine character representations into tokens has made training these models more…

Computation and Language · Computer Science 2023-11-16 William Fleshman , Benjamin Van Durme

An essential design decision for multilingual Neural Text-To-Speech (NTTS) systems is how to represent input linguistic features within the model. Looking at the wide variety of approaches in the literature, two main paradigms emerge,…

Audio and Speech Processing · Electrical Eng. & Systems 2022-07-05 Ariadna Sanchez , Alessio Falai , Ziyao Zhang , Orazio Angelini , Kayoko Yanagisawa

Recent trends in natural language processing research and annotation tasks affirm a paradigm shift from the traditional reliance on a single ground truth to a focus on individual perspectives, particularly in subjective tasks. In scenarios…

Computation and Language · Computer Science 2024-04-18 Olufunke O. Sarumi , Béla Neuendorf , Joan Plepi , Lucie Flek , Jörg Schlötterer , Charles Welch

Language-independent tokenisation (LIT) methods that do not require labelled language resources or lexicons have recently gained popularity because of their applicability in resource-poor languages. Moreover, they compactly represent a…

Computation and Language · Computer Science 2020-02-26 Danushka Bollegala , Ryuichi Kiryo , Kosuke Tsujino , Haruki Yukawa

We introduce the task of cross-lingual semantic parsing: mapping content provided in a source language into a meaning representation based on a target language. We present: (1) a meaning representation designed to allow systems to target…

Computation and Language · Computer Science 2018-04-24 Sheng Zhang , Kevin Duh , Benjamin Van Durme

Sign Language Translation has attained considerable success recently, raising hopes for improved communication with the Deaf. A pre-processing step called tokenization improves the success of translations. Tokens can be learned from sign…

Computer Vision and Pattern Recognition · Computer Science 2020-03-05 Alptekin Orbay , Lale Akarun

Recent Named Entity Recognition (NER) advancements have significantly enhanced text classification capabilities. This paper focuses on spoken NER, aimed explicitly at spoken document retrieval, an area not widely studied due to the lack of…

Computation and Language · Computer Science 2024-09-12 Moncef Benaicha , David Thulke , M. A. Tuğtekin Turan

Large pretrained language models (LMs) like BERT have improved performance in many disparate natural language processing (NLP) tasks. However, fine tuning such models requires a large number of training examples for each target task.…

Computation and Language · Computer Science 2022-01-28 Jixuan Wang , Kuan-Chieh Wang , Frank Rudzicz , Michael Brudno