English
Related papers

Related papers: An Interpretable and Crosslingual Method for Evalu…

200 papers

We describe a two-step approach for dialogue management in task-oriented spoken dialogue systems. A unified neural network framework is proposed to enable the system to first learn by supervision from a set of dialogue data and then…

Computation and Language · Computer Science 2016-06-09 Pei-Hao Su , Milica Gasic , Nikola Mrksic , Lina Rojas-Barahona , Stefan Ultes , David Vandyke , Tsung-Hsien Wen , Steve Young

This study investigates whether large language models (LLMs) exhibit cross-linguistic differences in mental health evaluations. Focusing on Chinese and English, we examine two widely used models, GPT-4o and Qwen3, to assess whether prompt…

Computation and Language · Computer Science 2026-03-10 Jiayi Xu , Xiyang Hu

Despite tremendous advancements in dialogue systems, stable evaluation still requires human judgments producing notoriously high-variance metrics due to their inherent subjectivity. Moreover, methods and labels in dialogue evaluation are…

Computation and Language · Computer Science 2023-08-01 Sarah E. Finch , James D. Finch , Jinho D. Choi

Pretrained multilingual models exhibit the same social bias as models processing English texts. This systematic review analyzes emerging research that extends bias evaluation and mitigation approaches into multilingual and non-English…

Computation and Language · Computer Science 2025-09-08 Lance Calvin Lim Gamboa , Yue Feng , Mark Lee

Large Language Models (LLMs) are increasingly deployed in high-stakes contexts where their outputs influence real-world decisions. However, evaluating bias in LLM outputs remains methodologically challenging due to sensitivity to prompt…

Computation and Language · Computer Science 2026-01-13 William Guey , Wei Zhang , Pei-Luen Patrick Rau , Pierrick Bougault , Vitor D. de Moura , Bertan Ucar , Jose O. Gomes

The longstanding goal of multi-lingual learning has been to develop a universal cross-lingual model that can withstand the changes in multi-lingual data distributions. There has been a large amount of work to adapt such multi-lingual models…

Computation and Language · Computer Science 2024-01-01 Meryem M'hamdi , Xiang Ren , Jonathan May

We introduce the task of cross-lingual semantic parsing: mapping content provided in a source language into a meaning representation based on a target language. We present: (1) a meaning representation designed to allow systems to target…

Computation and Language · Computer Science 2018-04-24 Sheng Zhang , Kevin Duh , Benjamin Van Durme

Reliable evaluation benchmarks designed for replicability and comprehensiveness have driven progress in machine learning. Due to the lack of a multilingual benchmark, however, vision-and-language research has mostly focused on English…

Computation and Language · Computer Science 2022-07-19 Emanuele Bugliarello , Fangyu Liu , Jonas Pfeiffer , Siva Reddy , Desmond Elliott , Edoardo Maria Ponti , Ivan Vulić

Vision-language models (VLMs) have demonstrated remarkable progress in multimodal reasoning. However, existing benchmarks remain limited in terms of high-quality, human-verified examples. Many current datasets rely on synthetically…

Computer Vision and Pattern Recognition · Computer Science 2025-11-13 Patrick Haller , Fabio Barth , Jonas Golde , Georg Rehm , Alan Akbik

Code-switching is a widespread practice among the world's multilingual majority, yet few benchmarks accurately reflect its complexity in everyday communication. We present PingPong, a benchmark for natural multi-party code-switching…

Cognitive processing signals can be used to improve natural language processing (NLP) tasks. However, it is not clear how these signals correlate with linguistic information. Bridging between human language processing and linguistic…

Computation and Language · Computer Science 2022-03-22 Yuqi Ren , Deyi Xiong

Image captioning has so far been explored mostly in English, as most available datasets are in this language. However, the application of image captioning should not be restricted by language. Only few studies have been conducted for image…

Computation and Language · Computer Science 2017-08-16 Weiyu Lan , Xirong Li , Jianfeng Dong

Clinical language processing has received a lot of attention in recent years, resulting in new models or methods for disease phenotyping, mortality prediction, and other tasks. Unfortunately, many of these approaches are tested under…

Computation and Language · Computer Science 2022-09-30 Travis R. Goodwin , Dina Demner-Fushman

Establishing stable mappings between natural language expressions and visual percepts is a foundational problem for both cognitive science and artificial intelligence. Humans routinely ground linguistic reference in noisy, ambiguous…

Artificial Intelligence · Computer Science 2026-02-24 Joseph Bingham

We propose an unsupervised method to obtain cross-lingual embeddings without any parallel data or pre-trained word embeddings. The proposed model, which we call multilingual neural language models, takes sentences of multiple languages as…

Computation and Language · Computer Science 2018-09-10 Takashi Wada , Tomoharu Iwata

Spoken Language Models (SLMs) aim to learn linguistic competence directly from speech using discrete units, widening access to Natural Language Processing (NLP) technologies for languages with limited written resources. However, progress…

Computation and Language · Computer Science 2026-02-23 Adel Moumen , Guangzhi Sun , Philip C. Woodland

The advent of NMT has expanded the scope of translation beyond isolated sentences, enabling context to be preserved across paragraphs and documents. However, current evaluation metrics largely remain restricted to the sentence level and…

Computational Engineering, Finance, and Science · Computer Science 2026-04-23 Hyeokmin Lee , Youngkyu Kim , Byounghyun Yoo

State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models. These models are generally trained on data in a single language (usually English), and cannot be directly used…

Computation and Language · Computer Science 2018-09-14 Alexis Conneau , Guillaume Lample , Ruty Rinott , Adina Williams , Samuel R. Bowman , Holger Schwenk , Veselin Stoyanov

Dialogue related Machine Reading Comprehension requires language models to effectively decouple and model multi-turn dialogue passages. As a dialogue development goes after the intentions of participants, its topic may not keep constant…

Computation and Language · Computer Science 2023-09-19 Xinbei Ma , Yi Xu , Hai Zhao , Zhuosheng Zhang

The remarkable understanding and generation capabilities of large language models (LLMs) have greatly improved translation performance. However, incorrect understanding of the sentence to be translated can degrade translation quality. To…

Computation and Language · Computer Science 2024-12-31 Andong Chen , Kehai Chen , Yang Xiang , Xuefeng Bai , Muyun Yang , Yang Feng , Tiejun Zhao , Min zhang