Sign Language Translation from Instructional Videos

Laia Tarrés; Gerard I. Gállego; Amanda Duarte; Jordi Torres; Xavier Giró-i-Nieto

Sign Language Translation from Instructional Videos

Computation and Language 2023-04-17 v2 Computer Vision and Pattern Recognition

Authors: Laia Tarrés , Gerard I. Gállego , Amanda Duarte , Jordi Torres , Xavier Giró-i-Nieto

Abstract

The advances in automatic sign language translation (SLT) to spoken languages have been mostly benchmarked with datasets of limited size and restricted domains. Our work advances the state of the art by providing the first baseline results on How2Sign, a large and broad dataset. We train a Transformer over I3D video features, using the reduced BLEU as a reference metric for validation, instead of the widely used BLEU score. We report a result of 8.03 on the BLEU score, and publish the first open-source implementation of its kind to promote further advances.

Keywords

sign language translation instructional videos How2Sign Transformer I3D features BLEU open-source

Cite

@article{arxiv.2304.06371,
  title  = {Sign Language Translation from Instructional Videos},
  author = {Laia Tarrés and Gerard I. Gállego and Amanda Duarte and Jordi Torres and Xavier Giró-i-Nieto},
  journal= {arXiv preprint arXiv:2304.06371},
  year   = {2023}
}

Comments

Paper accepted at WiCV @CVPR23

Sign Language Translation from Instructional Videos

Abstract

Keywords

Cite

Comments

Related papers