Metraipolis › archive › dc_ec2f79d2ac2d

Scaling Graph Reasoning in Language Models with Learnable Graph Positional Encodings — arXiv:2610.27439

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.27439
License (as archived)
CC0-1.0
Archived
2026-10-21T07:07:44.692Z
SHA-256 of the archived text
3dece0c1071d8018e3634cbdebefc25282262abb186df9d54c670edd15facaa3
Stable URL of this page
https://metraipolis.com/documento/dc_ec2f79d2ac2d

How to cite

Scaling Graph Reasoning in Language Models with Learnable Graph Positional Encodings — arXiv:2610.27439 — arXiv (en), https://arxiv.org/abs/2610.27439 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ec2f79d2ac2d (Metraipolis archive dc_ec2f79d2ac2d, sha256 3dece0c1071d8018e3634cbdebefc25282262abb186df9d54c670edd15facaa3)
BibTeX
@misc{dcec2f79d2ac2d,
  title = {Scaling Graph Reasoning in Language Models with Learnable Graph Positional Encodings — arXiv:2610.27439},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.27439},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ec2f79d2ac2d; sha256 3dece0c1071d8018e3634cbdebefc25282262abb186df9d54c670edd15facaa3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27439. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Solving reasoning tasks supported on graphs typically requires providing a Language Reasoning Model (LRM) with a serialized description of the graph. This approach results in the input length growing with the number of edges, which limits the size of the problems that can be solved. Furthermore, even when graphs fit in context, performance sharply decays as the graphs grow. We propose to address this by augmenting LRMs with scalable and permutation equivariant graph positional encodings that modulate attention logits. An additional post-fusion-style module incorporates structural information into hidden states. We compute these positional encodings using a GNN followed by a Graph Transformer. This eliminates the need for serialized adjacencies, reducing the context burden to $O(N)$. We jointly fine-tune the positional encoder and the LRM LoRA adapters using optimal solutions generated by graph-theoretic solvers, resulting in a computationally efficient method. Trained on small graphs, the model exhibits improved size-generalization across graph-theoretic tasks on graphs with up to 1,000 nodes, and produces fewer tokens than a conventional LRM. Additionally, we observe that our model remains robust to node relabelings. Evaluations on graph-theoretic benchmarks also show that performance on small graphs is preserved.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.27439 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:43.559Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).