In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users in writing mails by reducing repetitive typing. In the design and deployment of such a large-scale and complicated system, we faced several challenges including model selection, performance evaluation, serving and other practical issues. At the core of Smart Compose is a large-scale neural language model. We leveraged state-of-the-art machine learning techniques for language model training which enabled high-quality suggestion prediction, and constructed novel serving infrastructure for high-throughput and real-time inference. Experimental results show the effectiveness of our proposed system design and deployment approach. This system is currently being served in Gmail.
@article{arxiv.1906.00080,
title = {Gmail Smart Compose: Real-Time Assisted Writing},
author = {Mia Xu Chen and Benjamin N Lee and Gagan Bansal and Yuan Cao and Shuyuan Zhang and Justin Lu and Jackie Tsay and Yinan Wang and Andrew M. Dai and Zhifeng Chen and Timothy Sohn and Yonghui Wu},
journal= {arXiv preprint arXiv:1906.00080},
year = {2019}
}