English

Evaluating MT Systems: A Theoretical Framework

Computation and Language 2022-02-14 v1

Abstract

This paper outlines a theoretical framework using which different automatic metrics can be designed for evaluation of Machine Translation systems. It introduces the concept of {\em cognitive ease} which depends on {\em adequacy} and {\em lack of fluency}. Thus, cognitive ease becomes the main parameter to be measured rather than comprehensibility. The framework allows the components of cognitive ease to be broken up and computed based on different linguistic levels etc. Independence of dimensions and linearly combining them provides for a highly modular approach. The paper places the existing automatic methods in an overall framework, to understand them better and to improve upon them in future. It can also be used to evaluate the newer types of MT systems, such as speech to speech translation and discourse translation.

Keywords

Cite

@article{arxiv.2202.05806,
  title  = {Evaluating MT Systems: A Theoretical Framework},
  author = {Rajeev Sangal},
  journal= {arXiv preprint arXiv:2202.05806},
  year   = {2022}
}

Comments

18 pages

R2 v1 2026-06-24T09:32:35.846Z