Metraipolis › archive › dc_becc045cfdac

A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation — arXiv:2609.26527

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.26527
License (as archived): CC0-1.0
Archived: 2026-10-08T13:47:19.597Z
SHA-256 of the archived text: 42e3e656bda28759cb8ef2a03527db0e9bb36b706a927ad38ae445cea404afc9
Stable URL of this page: https://metraipolis.com/documento/dc_becc045cfdac

How to cite

A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation — arXiv:2609.26527 — arXiv (en), https://arxiv.org/abs/2609.26527 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_becc045cfdac (Metraipolis archive dc_becc045cfdac, sha256 42e3e656bda28759cb8ef2a03527db0e9bb36b706a927ad38ae445cea404afc9)

BibTeX

@misc{dcbecc045cfdac,
  title = {A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation — arXiv:2609.26527},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.26527},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_becc045cfdac; sha256 42e3e656bda28759cb8ef2a03527db0e9bb36b706a927ad38ae445cea404afc9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26527. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When two texts describe the same expression, standard metrics based on lexical overlap or whole-text similarity may fail to detect meaningful differences in how that expression is framed. We propose a framework to evaluate semiotic alignment between texts, where a semiotic profile encompasses both the contextual meaning and the discourse references made salient by a text. Our approach yields two scores, Semiotic Fidelity and Semiotic Coverage, estimating how much of one text's profile is supported by the other and how much of the other's profile it recovers. Experiments show that coverage is typically lower than fidelity, and that alignment between LLMs and human-curated data is highest at low sampling temperatures, while higher temperatures reduce this alignment.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.26527 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:18:38.326Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).