Metraipolis › archive › dc_9cc3bc78845f

HiddenPathQA: A Benchmark for Knowledge Graph Question Answering When Questions Hide Their Paths — arXiv:2611.11802

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.11802
License (as archived)
CC0-1.0
Archived
2026-11-10T18:43:41.399Z
SHA-256 of the archived text
2191fc286c09491d71032233c21fc2e6ebc75b4eb49e187be52545eba92164ad
Stable URL of this page
https://metraipolis.com/documento/dc_9cc3bc78845f

How to cite

HiddenPathQA: A Benchmark for Knowledge Graph Question Answering When Questions Hide Their Paths — arXiv:2611.11802 — arXiv (en), https://arxiv.org/abs/2611.11802 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_9cc3bc78845f (Metraipolis archive dc_9cc3bc78845f, sha256 2191fc286c09491d71032233c21fc2e6ebc75b4eb49e187be52545eba92164ad)
BibTeX
@misc{dc9cc3bc78845f,
  title = {HiddenPathQA: A Benchmark for Knowledge Graph Question Answering When Questions Hide Their Paths — arXiv:2611.11802},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.11802},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_9cc3bc78845f; sha256 2191fc286c09491d71032233c21fc2e6ebc75b4eb49e187be52545eba92164ad},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.11802. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Knowledge graph question answering (KGQA) systems are typically evaluated on questions whose surface tokens lexically expose the gold KG path. Users, however, may ask questions without knowing the underlying schema: "Which condition could Andrew have due to family history?" compresses a 4-hop chain "Andrew --[father]--> parent --[father]--> grandparent --[medical condition]--> disease --[subclass of]--> hereditary disorder" into a single phrase (family history) that names none of those relations. We identify this collapse of path-question alignment as a blind spot in existing KGQA evaluation. Three major LLM-KG paradigms, hop-by-hop exploration, plan-then-retrieve, and similarity-based retrieval, implicitly rely on such lexical cues, an assumption rarely tested in practice. We make three contributions. (i) We introduce HiddenPathQA, a manually verified benchmark of 1,055 questions spanning 2-6 hops and multiple domains, constructed so that question wording does not reveal the gold path's relations. (ii) We propose Holistic Path Selection, which compares questions against candidate relation chains in the local subgraph as complete units, replacing irreversible per-hop relation scoring with globally informed chain-level reasoning. We develop two implementations, TieredHolistic and FlatHolistic. (iii) We conduct a pseudoword stress test that replaces entity names with meaningless placeholders to distinguish KG-grounded reasoning from entity-level memorization. Holistic Path Selection outperforms six strong KGQA baselines (ToG, PoG, FiDeLiS, R2-KG, KAPING, and KARPA) by 8.8-17.9 accuracy points across two LLM backbones and both original and pseudoword settings. Its advantage under pseudoword replacement shows that the gains cannot be explained by entity-level memorization alone.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.11802 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.660Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).