English

Context-Augmented Code Generation: How Product Context Improves AI Coding Agent Decision Compliance by 49%

Software Engineering 2026-05-12 v1 Artificial Intelligence Computational Engineering, Finance, and Science Machine Learning Logic in Computer Science

Abstract

AI coding agents powered by large language models can read codebases and produce functional code, but they routinely violate team-specific product decisions that are invisible in the source code alone. We introduce a controlled benchmark measuring decision compliance, the rate at which an AI coding agent follows established product, design, and engineering decisions, across 8 realistic software engineering tasks containing 41 weighted decision points. We compare a baseline configuration (Claude Code with codebase access only) against an augmented configuration that adds Brief, a product-context retrieval system providing spec generation, mid-build consultation, and retrieval of recorded decisions, persona pain points, customer signals, and competitive intelligence. On identical prompts and the same repository, the augmented configuration achieves 95% decision compliance versus 46% for the baseline, a 49 percentage point improvement. Per-decision analysis reveals that the baseline achieves 100% compliance on decisions visible in the codebase and 0-33% on decisions requiring product context, suggesting that product-context retrieval is a key driver of the improvement. We release the benchmark repository, all 16 pull requests, and scoring harness for independent reproduction.

Keywords

Cite

@article{arxiv.2605.08112,
  title  = {Context-Augmented Code Generation: How Product Context Improves AI Coding Agent Decision Compliance by 49%},
  author = {Drew Dillon and Kasyap Varanasi},
  journal= {arXiv preprint arXiv:2605.08112},
  year   = {2026}
}

Comments

16 pages, 3 figures, 16 tables. Benchmark repository: https://github.com/brief-hq/dcbench

R2 v1 2026-07-01T12:58:22.697Z