Better Late Than Never：Simultaneous Speech-to-Text Translation Latency Metrics的元评估
计算与语言
2026-03-09 v2 人工智能
摘要
Simultaneous speech-to-text translation系统必须在翻译质量与延迟之间进行权衡。虽然质量评估已相当成熟，但延迟测量仍是一个挑战。现有指标在短格式设置和人工预分段情况下会产生不一致的结果。我们对延迟指标进行首次全面元评估，涵盖不同语言对和系统。我们发现当前指标与分段相关的结构性偏差。我们引入YAAL（Yet Another Average Lagging）用于更准确的短格式评估，以及LongYAAL用于无分段音频。我们提出SoftSegmenter，一种基于软词级对齐的再分段工具。我们表明YAAL和LongYAAL，以及SoftSegmenter，优于流行的延迟指标，使对短格式和长格式simultaneous speech translation系统的评估更可靠。我们在OmniSTEval工具包中实现了所有制品：https://github.com/pe-trik/OmniSTEval。
引用
@article{arxiv.2509.17349,
title = {Better Late Than Never: Meta-Evaluation of Latency Metrics for Simultaneous Speech-to-Text Translation},
author = {Peter Polák and Sara Papi and Luisa Bentivogli and Ondřej Bojar},
journal= {arXiv preprint arXiv:2509.17349},
year = {2026}
}
备注
Changes: - small change in the name (Evaluation -> Meta-Evaluation); - added reference to the implementation; - excluded two test sets (IWSLT22 En-Zh, En-Ja) because of incorrect and missing segmentation; - main results unchanged; - added Degenerate Policy Test; - added sensitivity of the metrics to change in the metric value