Metraipolis › archive › dc_1caef1c392af

Who's Keeping Score? Interactive Steering of LLM-Powered Scoring with Attune — arXiv:2608.14948

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.14948
License (as archived)
CC0-1.0
Archived
2026-10-18T19:34:19.643Z
SHA-256 of the archived text
24fa8214e39599cbc0f6ed1f40504c4bb87b36de2d216a1b58912e4c3e710dd4
Stable URL of this page
https://metraipolis.com/documento/dc_1caef1c392af

How to cite

Who's Keeping Score? Interactive Steering of LLM-Powered Scoring with Attune — arXiv:2608.14948 — arXiv (en), https://arxiv.org/abs/2608.14948 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1caef1c392af (Metraipolis archive dc_1caef1c392af, sha256 24fa8214e39599cbc0f6ed1f40504c4bb87b36de2d216a1b58912e4c3e710dd4)
BibTeX
@misc{dc1caef1c392af,
  title = {Who's Keeping Score? Interactive Steering of LLM-Powered Scoring with Attune — arXiv:2608.14948},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.14948},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1caef1c392af; sha256 24fa8214e39599cbc0f6ed1f40504c4bb87b36de2d216a1b58912e4c3e710dd4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14948. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) are increasingly used to score text records at scale (e.g., rating candidate resumes on a 1-5 scale). However, existing LLM-powered approaches do not account for the fact that effective scoring requires both holistic understanding of records and locally consistent judgments across similar ones. We present Attune, a mixed-initiative system for steerable LLM-powered scoring. Given a task description and scoring range, Attune performs pairwise comparisons across records to develop a global understanding first, and then resolves these comparisons into consistent score assignments-deriving scoring criteria and rules bottom-up in the process. These serve as shared representations of scoring logic that users can inspect and edit. Based on insights from a formative study (n = 12), Attune's interface introduces novel steering interactions that allow users to deterministically refine scoring logic. Users can provide examples, directly edit criteria, rules, or target distributions, and give natural language feedback-with all refinements compiling into constraints that guide re-scoring. We validate our approach through a technical evaluation across three workloads and a user study with domain experts (n = 8) in healthcare, law, education, and AI evaluation.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.HC).
Categorie arXiv: Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2608.14948 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T19:02:17.193Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).