AI coding agents can autonomously generate pull requests (PRs), yet little is known about how their contributions compare to those of humans. We analyze 33,596 agent-generated PRs (APRs) and 6,618 human PRs (HPRs) to compare code-change characteristics and message quality. We observe that APR-introduced symbols (functions and classes) are removed much sooner than those in HPRs (median time to removal 3 vs. 34 days) and are also removed more often (symbol churn 7.33% vs. 4.10%), reflecting a focus on other tasks like documentation and test updates. Agents generate stronger commit-level messages (semantic similarity 0.72 vs. 0.68) but lag humans at PR-level summarization (PR-commit similarity 0.86 vs. 0.88). Commit message length is the best predictor of description quality, indicating reliance on individual commits over full-PR reasoning. These findings highlight a gap between agents' micro-level precision and macro-level communication, suggesting opportunities to improve agent-driven development workflows.
Cite
@article{arxiv.2601.17627,
title = {Code Change Characteristics and Description Alignment: A Comparative Study of Agentic versus Human Pull Requests},
author = {Dung Pham and Taher A. Ghaleb},
journal= {arXiv preprint arXiv:2601.17627},
year = {2026}
}
Comments
Accepted at the 23rd International Conference on Mining Software Repositories (MSR '26)