Metraipolis › archive › dc_792ded784c14

Not What You Meant: Can LLMs Follow a Specified Negation Semantics? — arXiv:2609.27517

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.27517
License (as archived)
CC0-1.0
Archived
2026-10-08T09:11:25.811Z
SHA-256 of the archived text
ad1cfb9a671fcf88b027f6f75fbe6ce1db46ea8e4ce9dc2466d142f92f3bb900
Stable URL of this page
https://metraipolis.com/documento/dc_792ded784c14

How to cite

Not What You Meant: Can LLMs Follow a Specified Negation Semantics? — arXiv:2609.27517 — arXiv (en), https://arxiv.org/abs/2609.27517 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_792ded784c14 (Metraipolis archive dc_792ded784c14, sha256 ad1cfb9a671fcf88b027f6f75fbe6ce1db46ea8e4ce9dc2466d142f92f3bb900)
BibTeX
@misc{dc792ded784c14,
  title = {Not What You Meant: Can LLMs Follow a Specified Negation Semantics? — arXiv:2609.27517},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.27517},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_792ded784c14; sha256 ad1cfb9a671fcf88b027f6f75fbe6ce1db46ea8e4ce9dc2466d142f92f3bb900},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.27517. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Negation does not carry a uniform interpretation across domains. In legal, regulatory, and medical reasoning, the intended interpretation depends on the reading in force -- open- versus closed-world, two- versus three-valued, and credulous versus skeptical. We study which reading of negation large language models adopt by default and whether they can override that preference when a different reading is explicitly specified. To this end, we introduce NAFBench, a procedural generator of solver-certified instances spanning four semantic viewpoints: SLDNF, well-founded semantics (WFS), and credulous and skeptical reasoning under stable-model semantics. The generator emits ground normal logic programs with controlled depth, width, and cycle structure. Each program is solved under all four viewpoints using SWI-Prolog, a well-founded semantics solver, and clingo, yielding up to four divergent labels. The programs are then verbalized into natural language under multiple framings and rule orderings that leave the answer invariant. The results expose a consistent gap. Across open-source models, following a specified negation semantics remains unsolved: the strongest models score 59--74% across the four semantic viewpoints, while the weakest score 31--67%. All models are order-sensitive on more than half of logically identical rule shufflings, while the two weaker models frequently overcommit on well-founded "undefined." Two frontier models reach 100% on the main fixed-complexity evaluation set, and a third, o4-mini, is near-perfect, falling only to 81% on well-founded "undefined." Delegating reasoning to a solver, fine-tuning on certified traces, or forcing an explicit three-valued verdict each partly closes the gap.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.27517 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T08:13:26.516Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).