Metraipolis › archive › dc_79012bfee3c5

PaperScorer: How Predictable Is Paper Acceptance? — arXiv:2610.31791

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.31791
License (as archived)
CC0-1.0
Archived
2026-10-26T20:11:21.257Z
SHA-256 of the archived text
1b749449f44b9e7f6c3bb8bed5bb37a30aa34c8933a42a25693598d43e66a253
Stable URL of this page
https://metraipolis.com/documento/dc_79012bfee3c5

How to cite

PaperScorer: How Predictable Is Paper Acceptance? — arXiv:2610.31791 — arXiv (en), https://arxiv.org/abs/2610.31791 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_79012bfee3c5 (Metraipolis archive dc_79012bfee3c5, sha256 1b749449f44b9e7f6c3bb8bed5bb37a30aa34c8933a42a25693598d43e66a253)
BibTeX
@misc{dc79012bfee3c5,
  title = {PaperScorer: How Predictable Is Paper Acceptance? — arXiv:2610.31791},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.31791},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_79012bfee3c5; sha256 1b749449f44b9e7f6c3bb8bed5bb37a30aa34c8933a42a25693598d43e66a253},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31791. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Automated research is largely evaluated by benchmark performance, but the value of research rarely reduces to a single score. Humans judge a work's holistic contribution through peer review, which is hard to model because it is noisy and depends on many factors. However, what if, given only the paper, we could predict its peer-review decision directly? We introduce PaperScorer, a family of models that predict a paper's accept or reject decision from its extracted text or page screenshots, training only on the decision after removing author identity and formatting shortcuts. Vision models are the most predictive (68% accuracy on ICLR papers and 77% on ML papers from arXiv), beating text-only and frontier reasoning models. Unlike plain text, page images retain figures, layout, and presentation. This signal is robust, surviving smaller models and even compressing the whole paper into a single image. But accurate predictions are not evenly spread. PaperScorer is strongest on the lowest and highest rated papers, while papers rated in the middle stay unpredictable for our models and frontier systems alike. Additionally, its confidence is calibrated, so its most confident predictions are also its most accurate. Overall, PaperScorer shows how much of the decision a paper carries on its own, marking a first step toward AI-driven research evaluation.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.31791 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:49:03.359Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).