The broadening adoption of machine learning in the enterprise is increasing the pressure for strict governance and cost-effective performance, in particular for the common and consequential steps of model storage and inference. The RDBMS provides a natural starting point, given its mature infrastructure for fast data access and processing, along with support for enterprise features (e.g., encryption, auditing, high-availability). To take advantage of all of the above, we need to address a key concern: Can in-RDBMS scoring of ML models match (outperform?) the performance of dedicated frameworks? We answer the above positively by building Raven, a system that leverages native integration of ML runtimes (i.e., ONNX Runtime) deep within SQL Server, and a unified intermediate representation (IR) to enable advanced cross-optimizations between ML and DB operators. In this optimization space, we discover the most exciting research opportunities that combine DB/Compiler/ML thinking. Our initial evaluation on real data demonstrates performance gains of up to 5.5x from the native integration of ML in SQL Server, and up to 24x from cross-optimizations--we will demonstrate Raven live during the conference talk.
@article{arxiv.1911.00231,
title = {Extending Relational Query Processing with ML Inference},
author = {Konstantinos Karanasos and Matteo Interlandi and Doris Xin and Fotis Psallidas and Rathijit Sen and Kwanghyun Park and Ivan Popivanov and Supun Nakandal and Subru Krishnan and Markus Weimer and Yuan Yu and Raghu Ramakrishnan and Carlo Curino},
journal= {arXiv preprint arXiv:1911.00231},
year = {2019}
}