Metraipolis › archive › dc_ef44f745ec67

IROH: Insightful Ranking Of Humor using Multi-Stage Hybrid Retrieval with Rationale-Distilled LLM Judges for JOKER 2026 Track Task 1 English — arXiv:2609.15618

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.15618
License (as archived)
CC0-1.0
Archived
2026-10-15T23:17:16.965Z
SHA-256 of the archived text
70daed71b445bda1fa8e08c72ad5a586b91b12a88adaabeaa08c0bf3156dae64
Stable URL of this page
https://metraipolis.com/documento/dc_ef44f745ec67

How to cite

IROH: Insightful Ranking Of Humor using Multi-Stage Hybrid Retrieval with Rationale-Distilled LLM Judges for JOKER 2026 Track Task 1 English — arXiv:2609.15618 — arXiv (en), https://arxiv.org/abs/2609.15618 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef44f745ec67 (Metraipolis archive dc_ef44f745ec67, sha256 70daed71b445bda1fa8e08c72ad5a586b91b12a88adaabeaa08c0bf3156dae64)
BibTeX
@misc{dcef44f745ec67,
  title = {IROH: Insightful Ranking Of Humor using Multi-Stage Hybrid Retrieval with Rationale-Distilled LLM Judges for JOKER 2026 Track Task 1 English — arXiv:2609.15618},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.15618},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef44f745ec67; sha256 70daed71b445bda1fa8e08c72ad5a586b91b12a88adaabeaa08c0bf3156dae64},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15618. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Our team, VANGUARD, presents IROH (Insightful Ranking of Humor), a three-stage retrieval system for JOKER Task 1 English at CLEF 2026, achieving first place on the leaderboard with 0.6347 MAP. Our pipeline combines hybrid sparse-dense retrieval, cross-encoder reranking, and a LoRA-adapted Large Language Model judge ensemble. We employ Gemma 4 to generate query-aware rationales under two prompt strategies, generic and typed, and produce up to four types of structured hard negatives for training data construction. Through an ablation across three cross-encoder architectures, four dense embedders, and eight judge configurations, our key findings are threefold: (1) the rationale-distilled judge is the primary driver of ranking quality, whereas appending rationales to the first-stage index contributes negligibly; (2) structured hard negatives degrade generalisation in nearly all configurations despite inflating local validation scores; and (3) across the components we ablate, the lighter, better-calibrated model is competitive with or stronger than its larger counterpart, with the generic-rationale Qwen2.5-7B judge (0.6055 MAP) outperforming every Gemma-4-31B configuration, and the advantage of generic over typed rationales is concentrated almost entirely in the smaller model.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CL).
Categorie arXiv: Information Retrieval (cs.IR) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.15618 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T22:51:10.038Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).