Metraipolis › archive › dc_587d8cd5eaba

CQ4OE: A benchmark for assessing LLM-assisted ontology generation from competency questions — arXiv:2609.26029

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.26029
License (as archived): CC0-1.0
Archived: 2026-10-08T13:45:24.654Z
SHA-256 of the archived text: 70138d0713ede3aec018e1283b135a8108d5318aeb1140cdc9043adfc04a4331
Stable URL of this page: https://metraipolis.com/documento/dc_587d8cd5eaba

How to cite

CQ4OE: A benchmark for assessing LLM-assisted ontology generation from competency questions — arXiv:2609.26029 — arXiv (en), https://arxiv.org/abs/2609.26029 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_587d8cd5eaba (Metraipolis archive dc_587d8cd5eaba, sha256 70138d0713ede3aec018e1283b135a8108d5318aeb1140cdc9043adfc04a4331)

BibTeX

@misc{dc587d8cd5eaba,
  title = {CQ4OE: A benchmark for assessing LLM-assisted ontology generation from competency questions — arXiv:2609.26029},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.26029},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_587d8cd5eaba; sha256 70138d0713ede3aec018e1283b135a8108d5318aeb1140cdc9043adfc04a4331},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26029. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Ontology generation from Competency Questions (CQs) is a central yet labor-intensive phase of Ontology Engineering. While large language models (LLMs) offer promising automation capabilities, current evaluations remain fragmented. Task formulations are heterogeneous, gold standards often lack fine-grained CQ provenance, metrics conflate lexical overlap with structural and logical adequacy, and reference ontologies are not always explicitly designed around the evaluation CQs. Here, we address these limitations with CQ4OE, a benchmark for the systematic and reproducible evaluation of LLM-based ontology generation from CQs. For each ontology in the benchmark, we build a CQ-driven gold OWL ontology with explicit provenance linking each CQ to the classes, properties, and axioms required to answer it. From this resource, we define two complementary evaluation tasks. CQ2Term supports term-level evaluation of CQ-specific class and property prediction over 99 CQs, and CQ2Onto supports ontology-level evaluation over 118 CQs, including hierarchy, property modeling, and axiom-level structure. We demonstrate CQ4OE with experiments using nine LLMs under zero-shot, iterative, and multi-agent generation strategies, showing that LLMs recover explicit vocabulary terms more reliably than creating ontologies, particularly in property modeling, hierarchy construction, and axiom generation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.26029 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:18:38.132Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).