Towards Building Autonomous Data Services on Azure
Abstract
Modern cloud has turned data services into easily accessible commodities. With just a few clicks, users are now able to access a catalog of data processing systems for a wide range of tasks. However, the cloud brings in both complexity and opportunity. While cloud users can quickly start an application by using various data services, it can be difficult to configure and optimize these services to gain the most value from them. For cloud providers, managing every aspect of an ever-increasing set of data services, while meeting customer SLAs and minimizing operational cost is becoming more challenging. Cloud technology enables the collection of significant amounts of workload traces and system telemetry. With the progress in data science (DS) and machine learning (ML), it is feasible and desirable to utilize a data-driven, ML-based approach to automate various aspects of data services, resulting in the creation of autonomous data services. This paper presents our perspectives and insights on creating autonomous data services on Azure. It also covers the future endeavors we plan to undertake and unresolved issues that still need attention.
Keywords
Cite
@article{arxiv.2405.01813,
title = {Towards Building Autonomous Data Services on Azure},
author = {Yiwen Zhu and Yuanyuan Tian and Joyce Cahoon and Subru Krishnan and Ankita Agarwal and Rana Alotaibi and Jesús Camacho-Rodríguez and Bibin Chundatt and Andrew Chung and Niharika Dutta and Andrew Fogarty and Anja Gruenheid and Brandon Haynes and Matteo Interlandi and Minu Iyer and Nick Jurgens and Sumeet Khushalani and Brian Kroth and Manoj Kumar and Jyoti Leeka and Sergiy Matusevych and Minni Mittal and Andreas Mueller and Kartheek Muthyala and Harsha Nagulapalli and Yoonjae Park and Hiren Patel and Anna Pavlenko and Olga Poppe and Santhosh Ravindran and Karla Saur and Rathijit Sen and Steve Suh and Arijit Tarafdar and Kunal Waghray and Demin Wang and Carlo Curino and Raghu Ramakrishnan},
journal= {arXiv preprint arXiv:2405.01813},
year = {2024}
}
Comments
SIGMOD Companion of the 2023 International Conference on Management of Data. 2023