English

Autonomous QA Agent: A Retrieval-Augmented Framework for Reliable Selenium Script Generation

Software Engineering 2026-05-05 v1 Artificial Intelligence Machine Learning

Abstract

Software testing is critical in the software development lifecycle, yet translating requirements into executable test scripts remains manual and error-prone. While Large Language Models (LLMs) can generate code, they often hallucinate non-existent UI elements. We present the Autonomous QA Agent, a Retrieval-Augmented Generation (RAG) system that grounds Selenium script generation in project-specific documentation and HTML structure. By ingesting diverse formats (Markdown, PDF, HTML) into a vector database, our system retrieves relevant context before generation. Evaluation on 20 e-commerce test scenarios shows our RAG approach achieves 100% (20/20) syntax validity and 90% (18/20, 95% CI: [85%, 95%], p < 0.001) execution success, compared to 30% for standard LLM generation. While our evaluation is limited to a single domain, our method significantly reduces hallucinations by grounding generation in actual DOM structure, demonstrating RAG's potential for automated UI testing.

Keywords

Cite

@article{arxiv.2601.06034,
  title  = {Autonomous QA Agent: A Retrieval-Augmented Framework for Reliable Selenium Script Generation},
  author = {Dudekula Kasim Vali},
  journal= {arXiv preprint arXiv:2601.06034},
  year   = {2026}
}

Comments

13 figures, 3 tables

R2 v1 2026-07-01T08:58:06.489Z