AEMA: Verifiable Evaluation Framework for Trustworthy and Controlled Agentic LLM Systems
Abstract
Evaluating large language model (LLM)-based multi-agent systems remains a critical challenge, as these systems must exhibit reliable coordination, transparent decision-making, and verifiable performance across evolving tasks. Existing evaluation approaches often limit themselves to single-response scoring or narrow benchmarks, which lack stability, extensibility, and automation when deployed in enterprise settings at multi-agent scale. We present AEMA (Adaptive Evaluation Multi-Agent), a process-aware and auditable framework that plans, executes, and aggregates multi-step evaluations across heterogeneous agentic workflows under human oversight. Compared to a single LLM-as-a-Judge, AEMA achieves greater stability, human alignment, and traceable records that support accountable automation. Our results on enterprise-style agent workflows simulated using realistic business scenarios demonstrate that AEMA provides a transparent and reproducible pathway toward responsible evaluation of LLM-based multi-agent systems. Keywords Agentic AI, Multi-Agent Systems, Trustworthy AI, Verifiable Evaluation, Human Oversight
Cite
@article{arxiv.2601.11903,
title = {AEMA: Verifiable Evaluation Framework for Trustworthy and Controlled Agentic LLM Systems},
author = {YenTing Lee and Keerthi Koneru and Zahra Moslemi and Sheethal Kumar and Ramesh Radhakrishnan},
journal= {arXiv preprint arXiv:2601.11903},
year = {2026}
}
Comments
Workshop on W51: How Can We Trust and Control Agentic AI? Toward Alignment, Robustness, and Verifiability in Autonomous LLM Agents at AAAI 2026