English

Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment

Computation and Language 2018-05-01 v1
Authors: Gonzalo Iglesias , William Tambellini , Adrià De Gispert , Eva Hasler , Bill Byrne
View on arXiv ↗ PDF ↗

Abstract

We describe a batched beam decoding algorithm for NMT with LMBR n-gram posteriors, showing that LMBR techniques still yield gains on top of the best recently reported results with Transformers. We also discuss acceleration strategies for deployment, and the effect of the beam size and batching on memory and speed.

Cite

@article{arxiv.1804.11324,
  title  = {Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment},
  author = {Gonzalo Iglesias and William Tambellini and Adrià De Gispert and Eva Hasler and Bill Byrne},
  journal= {arXiv preprint arXiv:1804.11324},
  year   = {2018}
}

Comments

Proceedings of NAACL-HLT 2018

Related papers

View all related →
Computation and Language · Computer Science
Later-stage Minimum Bayes-Risk Decoding for Neural Machine Translation
Raphael Shu, Hideki Nakayama
2017-06-09
Computation and Language · Computer Science
Beam Search Strategies for Neural Machine Translation
Markus Freitag, Yaser Al-Onaizan
2018-12-19
Computation and Language · Computer Science
Exploring Recombination for Efficient Decoding of Neural Machine Translation
Zhisong Zhang, Rui Wang, Masao Utiyama, Eiichiro Sumita +1
2018-10-16
Artificial Intelligence · Computer Science
Accelerating LLM Inference with Staged Speculative Decoding
Benjamin Spector, Chris Re
2023-08-10
Computation and Language · Computer Science
Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation
Luyu Gao, Xinyi Wang, Graham Neubig
2020-10-06
Computation and Language · Computer Science
Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU
Jacob Devlin
2017-05-08
Machine Learning · Computer Science
The Synergy of Speculative Decoding and Batching in Serving Large Language Models
Qidong Su, Christina Giannoula, Gennady Pekhimenko
2023-10-31
Computation and Language · Computer Science
Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios
Luohe Shi, Zuchao Li, Lefei Zhang, Baoyuan Qi +2
2025-11-26
Computation and Language · Computer Science
Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Model
Christian Tomani, David Vilar, Markus Freitag, Colin Cherry +4
2024-07-12
Computation and Language · Computer Science
Asynchronous and Segmented Bidirectional Encoding for NMT
Jingpu Yang, Zehua Han, Mengyu Xiang, Helin Wang +2
2024-02-26
Machine Learning · Computer Science
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Yichao Fu, Peter Bailis, Ion Stoica, Hao Zhang
2024-02-06
Computation and Language · Computer Science
Making the Most of your Model: Methods for Finetuning and Applying Pretrained Transformers
Davis Yoshida
2024-08-30
Machine Learning · Computer Science
Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
Hayate Iso, Tiyasa Mitra, Sudipta Mondal, Rasoul Shafipour +14
2026-04-30
Computation and Language · Computer Science
Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation
Matt Post, David Vilar
2018-11-13
Computation and Language · Computer Science
Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation
Mathias Müller, Rico Sennrich
2021-05-19
Computation and Language · Computer Science
Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory
Ya Li, Xinyu Liu, Dan Liu, Xueqiang Zhang +1
2019-02-01
Computation and Language · Computer Science
Language-Informed Beam Search Decoding for Multilingual Machine Translation
Yilin Yang, Stefan Lee, Prasad Tadepalli
2024-08-13
Computation and Language · Computer Science
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation
Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li +7
2025-06-03
Numerical Analysis · Mathematics
On Extensions of Limited Memory Steepest Descent Method
Qinmeng Zou, Frederic Magoules
2019-12-10
Computation and Language · Computer Science
Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation
Boxuan Lyu, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura
2025-05-27
Computation and Language · Computer Science
Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Lattices
Felix Stahlberg, Adrià de Gispert, Eva Hasler, Bill Byrne
2017-02-14
Computation and Language · Computer Science
Combining SMT and NMT Back-Translated Data for Efficient NMT
Alberto Poncelas, Maja Popovic, Dimitar Shterionov, Gideon Maillette de Buy Wenniger +1
2019-09-10
Computation and Language · Computer Science
When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?
Ye Qi, Devendra Singh Sachan, Matthieu Felix, Sarguna Janani Padmanabhan +1
2018-04-19
Machine Learning · Computer Science
Progressive Mixed-Precision Decoding for Efficient LLM Inference
Hao Mark Chen, Fuwen Tan, Alexandros Kouris, Royson Lee +2
2025-02-28
Artificial Intelligence · Computer Science
Dynamic-Width Speculative Beam Decoding for Efficient LLM Inference
Zongyue Qin, Zifan He, Neha Prakriya, Jason Cong +1
2025-03-17
R2 v1 2026-06-23T01:40:24.091Z