English

Reviewing the Reviewer: Elevating Peer Review Quality through LLM-Guided Feedback

Computation and Language 2026-02-12 v1 Computers and Society

Abstract

Peer review is central to scientific quality, yet reliance on simple heuristics -- lazy thinking -- has lowered standards. Prior work treats lazy thinking detection as a single-label task, but review segments may exhibit multiple issues, including broader clarity problems, or specificity issues. Turning detection into actionable improvements requires guideline-aware feedback, which is currently missing. We introduce an LLM-driven framework that decomposes reviews into argumentative segments, identifies issues via a neurosymbolic module combining LLM features with traditional classifiers, and generates targeted feedback using issue-specific templates refined by a genetic algorithm. Experiments show our method outperforms zero-shot LLM baselines and improves review quality by up to 92.4\%. We also release LazyReviewPlus, a dataset of 1,309 sentences labeled for lazy thinking and specificity.

Keywords

Cite

@article{arxiv.2602.10118,
  title  = {Reviewing the Reviewer: Elevating Peer Review Quality through LLM-Guided Feedback},
  author = {Sukannya Purkayastha and Qile Wan and Anne Lauscher and Lizhen Qu and Iryna Gurevych},
  journal= {arXiv preprint arXiv:2602.10118},
  year   = {2026}
}

Comments

39 pages, 22 figures, 29 tables