Metraipolis › archive › dc_855ed5342867

Recipe-Matching, Not Equivalence — arXiv:2609.31927

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31927
License (as archived)
CC0-1.0
Archived
2026-10-07T21:53:52.544Z
SHA-256 of the archived text
7b47c7018f235f973090bdaff8fc12c99f9571dfc4b81cc92776abbef2a4409a
Stable URL of this page
https://metraipolis.com/documento/dc_855ed5342867

How to cite

Recipe-Matching, Not Equivalence — arXiv:2609.31927 — arXiv (en), https://arxiv.org/abs/2609.31927 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_855ed5342867 (Metraipolis archive dc_855ed5342867, sha256 7b47c7018f235f973090bdaff8fc12c99f9571dfc4b81cc92776abbef2a4409a)
BibTeX
@misc{dc855ed5342867,
  title = {Recipe-Matching, Not Equivalence — arXiv:2609.31927},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31927},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_855ed5342867; sha256 7b47c7018f235f973090bdaff8fc12c99f9571dfc4b81cc92776abbef2a4409a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31927. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

MathNet-Retrieve asks a retriever to find, for a math problem, a document stating the same problem. An LLM under one fixed prompt writes each gold document and its near-miss distractors; LLM judges filter them. We call this procedure the "recipe", training on pairs built the same way "recipe-matching", and ask how much score it buys beyond the ability the benchmark claims to test. Two models from one base, matched in rows and settings, differ only in the training file: pairs written under the benchmark's published prompt by another vendor's LLM and judge, or computer-algebra-verified pairs with no LLM anywhere. The first leads by 45 R@1 points on the easy tier. By a non-LLM paraphrase control, half to two thirds of that gap comes from the pairs being LLM-written at all: LLM rewrites under two unrelated prompts, with the verified model's negatives, recover 30 and 22 of the 45 points; back-translations with the same negatives recover almost none. The remaining 15 to 25 points appear only under the benchmark's own prompt and vanish on real duplicates no generator wrote, the same problem in two languages. The hard tier rewards the recipe's pair structure, a deep rewrite against a minimal-edit near-miss: LLM rewrites alone score zero on it, attaching negatives unlocks it, and every negative that does so costs cross-language points; the sets scoring highest on it separate near-misses no LLM wrote worse than LLM rewrites with verified negatives. MELD also moves when a model trains on pairs built its way, without losing retention; on SABER-Math the registered attack fails, and the one gain, from its LLM-written summaries, is small but holds at a matched budget. Only on MathNet-Retrieve could we pin an inversion, benchmark score up and real retention down, to one edit of a training file. We release the generator-free duplicate evaluations, the near-miss test and three trained models.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CL).
Categorie arXiv: Information Retrieval (cs.IR) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.31927 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T21:27:08.726Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).