Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction
Abstract
We introduce Graph-Augmented Sequence-to-Sequence (GA-S2S), a novel framework that integrates a T5-small encoder-decoder with a Relational Graph Attention Network (RGAT) to improve link prediction in knowledge graphs. While existing Seq2Seq models rely solely on surface-level textual descriptions of entities and relations and at best, flatten the neighborhoods of a query entity into a single linear sequence, thereby discarding the inherent graph structure, GA-S2S jointly encodes both textual features and the full -hop subgraph topology surrounding the query entity. By integrating raw encoder outputs with RGAT's relation-aware embeddings, our model captures and leverages richer multi-hop relational patterns and textual information. Our preliminary experiments on the CoDEx dataset demonstrate that GA-S2S outperforms competitive Seq2Seq-based baseline models, achieving up to a 19\% relative gain in link prediction accuracy.
Keywords
Cite
@article{arxiv.2605.18211,
title = {Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction},
author = {Luu Huu Phuc and Ratan Bahadur Thapa and Mojtaba Nayyeri and Jingcheng Wu and Evgeny Kharlamov and Steffen Staab},
journal= {arXiv preprint arXiv:2605.18211},
year = {2026}
}
Comments
9 pages, 1 figure, 2 tables. Preprint of a paper accepted at the 5th Workshop on LLM-Integrated Knowledge Graph Generation from Text (TEXT2KG), co-located with ESWC 2026, May 10--14, 2026, Dubrovnik, Croatia