English
Related papers

Related papers: Difficulty-Aware Machine Translation Evaluation

200 papers

Neural Machine Translation (NMT) is a new approach to machine translation that has made great progress in recent years. However, recent studies show that NMT generally produces fluent but inadequate translations (Tu et al. 2016b; Tu et al.…

Computation and Language · Computer Science 2017-01-02 Xing Wang , Zhengdong Lu , Zhaopeng Tu , Hang Li , Deyi Xiong , Min Zhang

Although neural machine translation (NMT) has advanced the state-of-the-art on various language pairs, the interpretability of NMT remains unsatisfactory. In this work, we propose to address this gap by focusing on understanding the…

Computation and Language · Computer Science 2019-09-18 Shilin He , Zhaopeng Tu , Xing Wang , Longyue Wang , Michael R. Lyu , Shuming Shi

With the fast development of Machine Translation (MT) systems, especially the new boost from Neural MT (NMT) models, the MT output quality has reached a new level of accuracy. However, many researchers criticised that the current popular…

Computation and Language · Computer Science 2022-11-11 Lifeng Han

Machine Translation (MT) tools are widely used today, often in contexts where professional translators are not present. Despite progress in MT technology, a gap persists between system development and real-world usage, particularly for…

Large language models (LLMs) have achieved strong performance in general machine translation, yet their ability in culture-aware scenarios remains poorly understood. To bridge this gap, we introduce CanMT, a Culture-Aware Novel-Driven…

Computation and Language · Computer Science 2026-04-28 Zekun Yuan , Yangfan Ye , Xiaocheng Feng , Baohang Li , Qichen Hong , Yunfei Lu , Dandan Tu , Bing Qin

Automatic evaluation of language generation systems is a well-studied problem in Natural Language Processing. While novel metrics are proposed every year, a few popular metrics remain as the de facto metrics to evaluate tasks such as image…

Computation and Language · Computer Science 2020-10-27 Ozan Caglayan , Pranava Madhyastha , Lucia Specia

Machine translation (MT) of user-generated content (UGC) poses unique challenges, including handling slang, emotion, and literary devices like irony and sarcasm. Evaluating the quality of these translations is challenging as current metrics…

Computation and Language · Computer Science 2024-10-07 Shenbin Qian , Constantin Orăsan , Diptesh Kanojia , Félix do Carmo

Neural Machine Translation (NMT) models have shown remarkable performance but remain largely opaque in their decision making processes. The interpretability of these models, especially their internal attention mechanisms, is critical for…

Artificial Intelligence · Computer Science 2024-12-30 Anurag Mishra

Machine translation (MT) is an important task in natural language processing (NLP) as it automates the translation process and reduces the reliance on human translators. With the resurgence of neural networks, the translation quality…

Computation and Language · Computer Science 2021-01-14 Sameen Maruf , Fahimeh Saleh , Gholamreza Haffari

Literary translation is a culturally significant task, but it is bottlenecked by the small number of qualified literary translators relative to the many untranslated works published around the world. Machine translation (MT) holds potential…

Computation and Language · Computer Science 2022-10-27 Katherine Thai , Marzena Karpinska , Kalpesh Krishna , Bill Ray , Moira Inghilleri , John Wieting , Mohit Iyyer

Recent machine translation (MT) metrics calibrate their effectiveness by correlating with human judgement but without any insights about their behaviour across different error types. Challenge sets are used to probe specific dimensions of…

Computation and Language · Computer Science 2024-01-30 Nikita Moghe , Arnisa Fazla , Chantal Amrhein , Tom Kocmi , Mark Steedman , Alexandra Birch , Rico Sennrich , Liane Guillou

Machine Translation (MT) has been widely used for cross-lingual classification, either by translating the test set into English and running inference with a monolingual model (translate-test), or translating the training set into the target…

Computation and Language · Computer Science 2023-05-24 Mikel Artetxe , Vedanuj Goswami , Shruti Bhosale , Angela Fan , Luke Zettlemoyer

Modern machine translation (MT) systems depend on large parallel corpora, often collected from the Internet. However, recent evidence indicates that (i) a substantial portion of these texts are machine-generated translations, and (ii) an…

Computation and Language · Computer Science 2025-11-06 Cristian García-Romero , Miquel Esplà-Gomis , Felipe Sánchez-Martínez

Translations systematically diverge from texts originally produced in the target language, a phenomenon widely referred to as translationese. Translationese has been attributed to production tendencies (e.g. interference, simplification),…

Computation and Language · Computer Science 2026-03-13 Maria Kunilovskaya

Large Language Models (LLMs) have significantly advanced Machine Translation (MT), applying them to linguistically complex domains-such as Social Network Services, literature etc. In these scenarios, translations often require handling…

Computation and Language · Computer Science 2026-04-17 Yanzhi Tian , Cunxiang Wang , Zeming Liu , Heyan Huang , Wenbo Yu , Dawei Song , Jie Tang , Yuhang Guo

We reassess a recent study (Hassan et al., 2018) that claimed that machine translation (MT) has reached human parity for the translation of news from Chinese into English, using pairwise ranking and considering three variables that were not…

Computation and Language · Computer Science 2018-08-31 Antonio Toral , Sheila Castilho , Ke Hu , Andy Way

For the past 60 years, Research in machine translation is going on. For the development in this field, a lot of new techniques are being developed each day. As a result, we have witnessed development of many automatic machine translators. A…

Computation and Language · Computer Science 2013-07-25 Nisheeth Joshi , Hemant Darbari , Iti Mathur

We present an interactive machine translation (MT) system designed for users who are not proficient in the target language. It aims to improve trustworthiness and explainability by identifying potentially mistranslated words and allowing…

Computation and Language · Computer Science 2025-04-01 Kenneth J. Sible , David Chiang

Neural machine translation (NMT) systems amplify lexical biases present in their training data, leading to artificially impoverished language in output translations. These language-level characteristics render automatic translations…

Computation and Language · Computer Science 2025-06-02 Huiyuan Lai , Esther Ploeger , Rik van Noord , Antonio Toral

Evaluation plays a vital role in checking the quality of MT output. It is done either manually or automatically. Manual evaluation is very time consuming and subjective, hence use of automatic metrics is done most of the times. This paper…

Computation and Language · Computer Science 2015-06-19 Aditi Kalyani , Hemant Kumud , Shashi Pal Singh , Ajai Kumar