Metraipolis › archive › dc_395f320404cb
The illusion of neutrality in metric-based research evaluation — arXiv:2609.24471
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.24471
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:59:45.167Z
- SHA-256 of the archived text
0a26cf4c06bd037913b80d9f3659cebaa381aaa9e49b48f5ec6fe605503781ec- Stable URL of this page
https://metraipolis.com/documento/dc_395f320404cb
How to cite
The illusion of neutrality in metric-based research evaluation — arXiv:2609.24471 — arXiv (en), https://arxiv.org/abs/2609.24471 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_395f320404cb (Metraipolis archive dc_395f320404cb, sha256 0a26cf4c06bd037913b80d9f3659cebaa381aaa9e49b48f5ec6fe605503781ec)
BibTeX
@misc{dc395f320404cb,
title = {The illusion of neutrality in metric-based research evaluation — arXiv:2609.24471},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.24471},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_395f320404cb; sha256 0a26cf4c06bd037913b80d9f3659cebaa381aaa9e49b48f5ec6fe605503781ec},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24471. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Venue prestige and citation counts are two widely used, albeit imperfect, signals of research quality. When the two signals conflict, evaluators must decide how much weight to assign each. Yet, it remains unknown how researchers across disciplines trade off these two signals when evaluating research outcome. To fill this gap, we surveyed 869 researchers using paired choices between hypothetical departmental hiring rules that assigned different weights to venue prestige and citation counts, asking which would produce better science. Among 795 respondents whose choices were largely internally consistent, choices placed nearly equal aggregate weight on the two signals. This apparent balance concealed substantial individual heterogeneity: nearly two in five respondents occupied the most venue-heavy or citation-heavy intervals. Moreover, respondents on average chose more citation-heavy rules than they believed their departments used in hiring. By revealing the subjective judgments that arise when research indicators conflict, our findings reinforce calls for greater caution when quantitative indicators are used to evaluate research. Tema degli esploratori: ricerca e scienza aperta (retro-catalogo arXiv: annunci del 2026-09-22 in cs.DL). Categorie arXiv: Digital Libraries (cs.DL). Pagina del lavoro: https://arxiv.org/abs/2609.24471 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:53:12.669Z dalla pagina https://arxiv.org/catchup/cs.DL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).