We present OpenSeq2Seq - a TensorFlow-based toolkit for training sequence-to-sequence models that features distributed and mixed-precision training. Benchmarks on machine translation and speech recognition tasks show that models built using OpenSeq2Seq give state-of-the-art performance at 1.5-3x less training time. OpenSeq2Seq currently provides building blocks for models that solve a wide range of tasks including neural machine translation, automatic speech recognition, and speech synthesis.
@article{arxiv.1805.10387,
title = {Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq},
author = {Oleksii Kuchaiev and Boris Ginsburg and Igor Gitman and Vitaly Lavrukhin and Jason Li and Huyen Nguyen and Carl Case and Paulius Micikevicius},
journal= {arXiv preprint arXiv:1805.10387},
year = {2018}
}
Comments
Presented at Workshop for Natural Language Processing Open Source Software (NLP-OSS), co-located with ACL2018