C-NMT: A Collaborative Inference Framework for Neural Machine Translation
Machine Learning
2022-04-11 v1 Artificial Intelligence
Computation and Language
Systems and Control
Systems and Control
Abstract
Collaborative Inference (CI) optimizes the latency and energy consumption of deep learning inference through the inter-operation of edge and cloud devices. Albeit beneficial for other tasks, CI has never been applied to the sequence- to-sequence mapping problem at the heart of Neural Machine Translation (NMT). In this work, we address the specific issues of collaborative NMT, such as estimating the latency required to generate the (unknown) output sequence, and show how existing CI methods can be adapted to these applications. Our experiments show that CI can reduce the latency of NMT by up to 44% compared to a non-collaborative approach.
Cite
@article{arxiv.2204.04043,
title = {C-NMT: A Collaborative Inference Framework for Neural Machine Translation},
author = {Yukai Chen and Roberta Chiaro and Enrico Macii and Massimo Poncino and Daniele Jahier Pagliari},
journal= {arXiv preprint arXiv:2204.04043},
year = {2022}
}
Comments
Accepted as a conference paper at the 2022 IEEE International Symposium on Circuits and Systems (ISCAS)