English

Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing

Computation and Language 2026-05-21 v2

Abstract

Knowledge editing is pivotal for efficiently updating the parametric memory of Large Language Models (LLMs), enabling them to function as evolving agents in dynamic environments. However, mainstream in-parameter knowledge editing approaches suffer from Subject-Dominant Memory Interference: modifying a specific fact inadvertently corrupts the broader structural knowledge associated with the same subject within LLMs. We diagnose the root cause as a shortcut learning pathology, where the optimization objective overfits subject representations while bypassing the essential relational context. To rectify this, we propose Causal Path Alignment (CPA), a principled framework designed to anchor the optimization trajectory to valid causal pathways. CPA enforces parameter updates to route through relation-aware intermediate states, thereby preventing the erasure of contextual dependencies. Experimental results across diverse LLM backbones demonstrate that CPA consistently eliminates the shortcut, significantly improving relation specificity while exhibiting minimal side-effects. Moreover, CPA serves as a model-agnostic plug-in for existing editors, paving the way for reliable and trustworthy in-parameter knowledge editing.

Keywords

Cite

@article{arxiv.2506.04042,
  title  = {Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing},
  author = {Xiyu Liu and Zhengxiao Liu and Naibin Gu and Zheng Lin and Weiping Wang},
  journal= {arXiv preprint arXiv:2506.04042},
  year   = {2026}
}

Comments

Accepted by IJCAI 2026

R2 v1 2026-07-01T02:59:12.982Z