Metraipolis › archive › dc_d1a52e4da3cb

NEPSEE: A Double-Annotated Benchmark for Nepali Sentiment Analysis — arXiv:2610.19190

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.19190
License (as archived)
CC0-1.0
Archived
2026-10-15T20:23:38.246Z
SHA-256 of the archived text
236727f248df355ae3078f53f04a8c75251151eec2e6a5c613448c12b1e75010
Stable URL of this page
https://metraipolis.com/documento/dc_d1a52e4da3cb

How to cite

NEPSEE: A Double-Annotated Benchmark for Nepali Sentiment Analysis — arXiv:2610.19190 — arXiv (en), https://arxiv.org/abs/2610.19190 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d1a52e4da3cb (Metraipolis archive dc_d1a52e4da3cb, sha256 236727f248df355ae3078f53f04a8c75251151eec2e6a5c613448c12b1e75010)
BibTeX
@misc{dcd1a52e4da3cb,
  title = {NEPSEE: A Double-Annotated Benchmark for Nepali Sentiment Analysis — arXiv:2610.19190},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.19190},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d1a52e4da3cb; sha256 236727f248df355ae3078f53f04a8c75251151eec2e6a5c613448c12b1e75010},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19190. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Many sentiment datasets use one annotator for each item, so the test labels show the conventions of that person. We build a Nepali benchmark from two annotators who labelled the same sentences blind. They labelled 5,614 sentences from tweets of Nepali public figures on a five-point scale. We keep only the 3,189 sentences where both give the same polarity. On the three-class benchmark, NepBERTa with a fixed tokenizer gets a macro-F1 of 0.877 and IndicBERTv2 0.867. Three commercial large language models (LLMs) get 0.882 to 0.896 zero-shot, and these differences are not significant. On the five-point scale, 766 sentences have two adjacent labels, for example Pos and SPos. We accept both labels and train on both. On this task, Claude Opus 5.5 beats the best encoders by 0.10 to 0.12 macro-F1. A tokenizer audit adds a second result. The default tokenizers of three Nepali BERT models delete the non-spacing Devanagari vowel signs and the virama. When we restore the marks, macro-F1 increases by 0.084 for NepBERTa but decreases by 0.095 for a model pretrained on stripped text.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.19190 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T19:36:18.173Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).