English

Compositional Consistency-Guided Decoding for Three-Way Logical Question Answering

Computation and Language 2026-05-28 v2 Artificial Intelligence Logic in Computer Science

Abstract

Three-way logical question answering (QA) assigns one of True\text{True}, False\text{False}, or Unknown\text{Unknown} to a hypothesis HH given a premise set SS. We study this task as a compact compositional inference problem: predictions for HH and for a mechanically negated hypothesis ¬H\neg H should agree under a deterministic negation map. Despite this simple structure, large language models (LLMs) can exhibit two practical failure modes: (i) negation inconsistency, where answers to HH and ¬H\neg H violate the required label mapping, and (ii) epistemic Unknown\text{Unknown}, where the model abstains even when one side is entailed. We introduce CGD-PD, a lightweight, training-free test-time layer that combines neural 3-way classification, symbolic negation-consistency projection, and targeted binary entailment probes. On one validation split of FOLIO's first-order logic fields, CGD-PD improves accuracy by 4.4 points on GPT-5.2 and 6.8 points on Claude Sonnet 4.5, while reducing Unknown\text{Unknown} predictions and epistemic abstention. These results provide a controlled proof of concept that simple logical composition at inference time can help evaluate and improve LLM reasoning reliability; they do not, by themselves, establish robustness beyond this formal benchmark setting.

Keywords

Cite

@article{arxiv.2604.06196,
  title  = {Compositional Consistency-Guided Decoding for Three-Way Logical Question Answering},
  author = {Tianyi Huang and Ming Hou and Jiaheng Su and Yutong Zhang and Ziling Zhang},
  journal= {arXiv preprint arXiv:2604.06196},
  year   = {2026}
}

Comments

Accepted at the ICML 2026 Workshop on Compositional Learning: Safety, Interpretability, and Agents

R2 v1 2026-07-01T11:57:55.989Z