Evaluation and Continual Improvement for an Enterprise AI Assistant
Human-Computer Interaction
2024-12-10 v2
Abstract
The development of conversational AI assistants is an iterative process with multiple components. As such, the evaluation and continual improvement of these assistants is a complex and multifaceted problem. This paper introduces the challenges in evaluating and improving a generative AI assistant for enterprises, which is under active development, and how we address these challenges. We also share preliminary results and discuss lessons learned.
Cite
@article{arxiv.2407.12003,
title = {Evaluation and Continual Improvement for an Enterprise AI Assistant},
author = {Akash V. Maharaj and Kun Qian and Uttaran Bhattacharya and Sally Fang and Horia Galatanu and Manas Garg and Rachel Hanessian and Nishant Kapoor and Ken Russell and Shivakumar Vaithyanathan and Yunyao Li},
journal= {arXiv preprint arXiv:2407.12003},
year = {2024}
}
Comments
Accepted to DaSH Workshop at NAACL 2024