Metraipolis › archive › dc_ef44f745ec67
IROH: Insightful Ranking Of Humor using Multi-Stage Hybrid Retrieval with Rationale-Distilled LLM Judges for JOKER 2026 Track Task 1 English — arXiv:2609.15618
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.15618
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T23:17:16.965Z
- SHA-256 of the archived text
70daed71b445bda1fa8e08c72ad5a586b91b12a88adaabeaa08c0bf3156dae64- Stable URL of this page
https://metraipolis.com/documento/dc_ef44f745ec67
How to cite
IROH: Insightful Ranking Of Humor using Multi-Stage Hybrid Retrieval with Rationale-Distilled LLM Judges for JOKER 2026 Track Task 1 English — arXiv:2609.15618 — arXiv (en), https://arxiv.org/abs/2609.15618 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef44f745ec67 (Metraipolis archive dc_ef44f745ec67, sha256 70daed71b445bda1fa8e08c72ad5a586b91b12a88adaabeaa08c0bf3156dae64)
BibTeX
@misc{dcef44f745ec67,
title = {IROH: Insightful Ranking Of Humor using Multi-Stage Hybrid Retrieval with Rationale-Distilled LLM Judges for JOKER 2026 Track Task 1 English — arXiv:2609.15618},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.15618},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef44f745ec67; sha256 70daed71b445bda1fa8e08c72ad5a586b91b12a88adaabeaa08c0bf3156dae64},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.15618. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Our team, VANGUARD, presents IROH (Insightful Ranking of Humor), a three-stage retrieval system for JOKER Task 1 English at CLEF 2026, achieving first place on the leaderboard with 0.6347 MAP. Our pipeline combines hybrid sparse-dense retrieval, cross-encoder reranking, and a LoRA-adapted Large Language Model judge ensemble. We employ Gemma 4 to generate query-aware rationales under two prompt strategies, generic and typed, and produce up to four types of structured hard negatives for training data construction. Through an ablation across three cross-encoder architectures, four dense embedders, and eight judge configurations, our key findings are threefold: (1) the rationale-distilled judge is the primary driver of ranking quality, whereas appending rationales to the first-stage index contributes negligibly; (2) structured hard negatives degrade generalisation in nearly all configurations despite inflating local validation scores; and (3) across the components we ablate, the lighter, better-calibrated model is competitive with or stronger than its larger counterpart, with the generic-rationale Qwen2.5-7B judge (0.6055 MAP) outperforming every Gemma-4-31B configuration, and the advantage of generic over typed rationales is concentrated almost entirely in the smaller model. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.CL). Categorie arXiv: Information Retrieval (cs.IR) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.15618 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:51:10.038Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).