Metraipolis › archive › dc_f2a2e8022499

What Gets Measured Gets Managed: Sign-aware Recommendation Needs Sign-aware Evaluation — arXiv:2609.33346

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33346
License (as archived)
CC0-1.0
Archived
2026-10-07T23:49:49.503Z
SHA-256 of the archived text
59e3751feb4954e32dd641c653543be9650382cb2e826324a7d925e9cd9cf785
Stable URL of this page
https://metraipolis.com/documento/dc_f2a2e8022499

How to cite

What Gets Measured Gets Managed: Sign-aware Recommendation Needs Sign-aware Evaluation — arXiv:2609.33346 — arXiv (en), https://arxiv.org/abs/2609.33346 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f2a2e8022499 (Metraipolis archive dc_f2a2e8022499, sha256 59e3751feb4954e32dd641c653543be9650382cb2e826324a7d925e9cd9cf785)
BibTeX
@misc{dcf2a2e8022499,
  title = {What Gets Measured Gets Managed: Sign-aware Recommendation Needs Sign-aware Evaluation — arXiv:2609.33346},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33346},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f2a2e8022499; sha256 59e3751feb4954e32dd641c653543be9650382cb2e826324a7d925e9cd9cf785},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33346. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Sign-aware recommender systems have recently been developed to leverage negative feedback for a deeper understanding of user preferences. However, our empirical diagnosis reveals that state-of-the-art graph-based sign-aware recommender systems are paradoxically valence-blind. Even though they explicitly incorporate sign information during training, they consistently fail to differentiate liked items from disliked ones at the ranking stage, frequently infiltrating top-K recommendations with disliked content. Through linear probing, we show that while valence information exists in the learned embeddings, it remains inaccessible to the inner-product scoring function. This widespread failure remains entirely undetected because conventional evaluation metrics, such as Recall, HR, and NDCG, assign a uniform utility of zero to both negative and unobserved items, creating a systematic evaluation blind spot. To bridge this gap, we propose a family of signed metrics, Signed Recall, Signed HR, and Signed NDCG, that explicitly penalize the recommendation of disliked content. Systematic re-evaluation under our proposed metrics fundamentally reshapes the established performance landscape, revealing that methods ranked highly under conventional metrics often fail to protect users from disliked content. Finally, through a proof-of-concept auxiliary loss, we confirm that the proposed metrics provide actionable training signals, guiding models toward valence-aware behavior without sacrificing conventional relevance. For transparency, our source code is available at: this https URL

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-29 in cs.IR).
Categorie arXiv: Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2609.33346 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T23:26:57.189Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).