Metraipolis › archive › dc_bce3f2d0b4ac

QuanReview: Offline, Auditable Reconciliation of Human and LLM Span Annotations — arXiv:2609.35685

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35685
License (as archived)
CC0-1.0
Archived
2026-10-07T21:53:48.212Z
SHA-256 of the archived text
700369dcd4e58af14e77c13e959a9d5333b2b5e9cfeecc916c7368d6a4fc5bae
Stable URL of this page
https://metraipolis.com/documento/dc_bce3f2d0b4ac

How to cite

QuanReview: Offline, Auditable Reconciliation of Human and LLM Span Annotations — arXiv:2609.35685 — arXiv (en), https://arxiv.org/abs/2609.35685 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bce3f2d0b4ac (Metraipolis archive dc_bce3f2d0b4ac, sha256 700369dcd4e58af14e77c13e959a9d5333b2b5e9cfeecc916c7368d6a4fc5bae)
BibTeX
@misc{dcbce3f2d0b4ac,
  title = {QuanReview: Offline, Auditable Reconciliation of Human and LLM Span Annotations — arXiv:2609.35685},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35685},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bce3f2d0b4ac; sha256 700369dcd4e58af14e77c13e959a9d5333b2b5e9cfeecc916c7368d6a4fc5bae},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35685. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Structured span annotations, such as quantities with their units, uncertainty modifiers, and event classes, are expensive to create and hard to keep trustworthy once language models enter the loop. We present QuanReview, an open-source system for auditing and correcting such annotation layers. QuanReview aligns two annotation streams over the same documents at character level, resolves unambiguous cases by an explicit and logged policy, and routes candidate conflicts to a browser-based adjudication interface where reviewers accept either side, build field-level hybrids, or flag items for re-annotation. A campaign manager assigns documents to multiple annotators with configurable redundancy, computes agreement at document and span level, auto-merges unanimous documents, and exports the corrected layer in the original file format, so that it can replace the original annotation files directly. Applied to a 4,457-record humanitarian benchmark and an LLM extraction stream, the system fully auto-merged 8% of documents, applied automatic policy decisions to a further 1,513 records, and concentrated human attention on 3,131 candidate conflicts, a mean of 5.4 per reviewed document.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.35685 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:27:08.702Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).