Large language models (LLMs) are highly sensitive to prompts, but most automatic prompt optimization (APO) methods assume access to ground-truth references (e.g., labeled validation data) that are costly to obtain. We propose the Prompt Duel Optimizer (PDO), a sample-efficient framework for label-free prompt optimization based on pairwise preference feedback from an LLM judge. PDO casts prompt selection as a dueling-bandit problem and combines (i) Double Thompson Sampling to prioritize informative comparisons under a fixed judge budget, with (ii) top-performer guided mutation to expand the candidate pool while pruning weak prompts. Experiments on BIG-bench Hard (BBH) and MS MARCO show that PDO consistently identifies stronger prompts than label-free baselines, while offering favorable quality--cost trade-offs under constrained comparison budgets.
@article{arxiv.2510.13907,
title = {LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization},
author = {Yuanchen Wu and Saurabh Verma and Justin Lee and Fangzhou Xiong and Poppy Zhang and Amel Awadelkarim and Xu Chen and Yubai Yuan and Shawndra Hill},
journal= {arXiv preprint arXiv:2510.13907},
year = {2026}
}
Comments
Accepted to Findings of ACL 2026. Camera-ready version