English

Knowledge-Guided Synthetic Bug Feedback for LLM-Based Unit Test Generation

Software Engineering 2026-07-13 v1

Abstract

Large language models (LLMs) have opened new opportunities for unit test generation, but executable tests do not necessarily reveal real defects. This paper studies how historical real-bug mechanisms can be transformed into executable feedback targets for LLM-based unit test generation. The proposed framework constructs structural and semantic representations of real-bug records, retrieves mechanisms applicable to a focal method, and instantiates them as synthetic bugs that guide iterative test enhancement. We evaluate the approach on method-level real-bug detection tasks from Defects4J and show that mechanism-guided synthetic-bug feedback improves real-bug detection over execution-, coverage-, mutation-, knowledge-, and search-based baselines. The results suggest that organizing real-bug mechanisms as retrievable and executable feedback targets is an effective way to guide generated tests toward bug-triggering inputs and behavioral oracles.

Cite

@article{arxiv.2607.11573,
  title  = {Knowledge-Guided Synthetic Bug Feedback for LLM-Based Unit Test Generation},
  author = {Ziheng Wang and Maike Li and Chen Zhi},
  journal= {arXiv preprint arXiv:2607.11573},
  year   = {2026}
}

Comments

12 pages, 7 figures, 6 tables. Preprint