Metraipolis › archive › dc_ca1cce2049ee
IntLawNER: A Named Entity Recognition Dataset and Benchmark in International Law — arXiv:2609.22529
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22529
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T21:58:51.383Z
- SHA-256 of the archived text
6eeffb3d536522f4aac14659a2881d909bc8bc202d52ad5d628bd56108921a73- Stable URL of this page
https://metraipolis.com/documento/dc_ca1cce2049ee
How to cite
IntLawNER: A Named Entity Recognition Dataset and Benchmark in International Law — arXiv:2609.22529 — arXiv (en), https://arxiv.org/abs/2609.22529 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ca1cce2049ee (Metraipolis archive dc_ca1cce2049ee, sha256 6eeffb3d536522f4aac14659a2881d909bc8bc202d52ad5d628bd56108921a73)
BibTeX
@misc{dcca1cce2049ee,
title = {IntLawNER: A Named Entity Recognition Dataset and Benchmark in International Law — arXiv:2609.22529},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22529},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ca1cce2049ee; sha256 6eeffb3d536522f4aac14659a2881d909bc8bc202d52ad5d628bd56108921a73},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22529. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
International law provides the normative framework through which states coordinate action, regulate armed conflict, and protect human rights, yet its texts remain without token-level named entity recognition (NER) resources. We introduce IntLawNER, a NER dataset and benchmark for codified sources of international law, covering 2,987 gold-annotated sentences and 8,094 entity spans from International Court of Justice (ICJ) decisions, UN Security Council resolutions, and European Court of Human Rights (ECtHR) judgments, annotated with seven institution-specific entity types. We construct IntLawNER with a cost-effective hybrid algorithmic-agentic pipeline that reduces 468k source sentences to a compact annotation set through candidate retrieval, LLM-based vetting, and human review, with 89.6% of gold spans accepted unchanged from the silver layer. However, the silver-to-gold analysis reveals that human-machine aggregate agreement metrics can be misleading in domain-specific NER: Cohen's kappa=0.964 on boundary-matched spans masks a macro-F1 of 0.753 when missing entities, boundary errors, and label corrections are included. The benchmark shows that zero-shot span-based GLiNER collapses on entity types dependent on institutional function rather than surface form (0.243 micro-F1), while fine-tuned transformers struggle on rare labels. Carefully selected few-shot examples that demonstrate label contrasts improve every LLM over zero-shot prompting, with Claude Opus 4.6 reaching the best score of 0.873 micro-F1. We release IntLawNER as a benchmark and reusable resource for extracting references in international legal texts. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.IR). Categorie arXiv: Artificial Intelligence (cs.AI) ; Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2609.22529 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T21:45:02.230Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).