Metraipolis › archive › dc_2bb26d1980db

From Granular Revision Operations to Meaningful Revision Units: Evaluating LLMs for Revision Boundary Detection — arXiv:2609.33720

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.33720
License (as archived): CC0-1.0
Archived: 2026-10-07T21:01:41.811Z
SHA-256 of the archived text: 7bc9f63234c89652af1c563977407aa29574ff6a13a0c70b123ba32645212311
Stable URL of this page: https://metraipolis.com/documento/dc_2bb26d1980db

How to cite

From Granular Revision Operations to Meaningful Revision Units: Evaluating LLMs for Revision Boundary Detection — arXiv:2609.33720 — arXiv (en), https://arxiv.org/abs/2609.33720 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2bb26d1980db (Metraipolis archive dc_2bb26d1980db, sha256 7bc9f63234c89652af1c563977407aa29574ff6a13a0c70b123ba32645212311)

BibTeX

@misc{dc2bb26d1980db,
  title = {From Granular Revision Operations to Meaningful Revision Units: Evaluating LLMs for Revision Boundary Detection — arXiv:2609.33720},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33720},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2bb26d1980db; sha256 7bc9f63234c89652af1c563977407aa29574ff6a13a0c70b123ba32645212311},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33720. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Revision traces provide valuable evidence about students' writing processes, but their usefulness for learning analytics depends on how individual revisions are represented. Automated draft-comparison methods often produce granular edit operations that can fragment a single purposeful revision into multiple analytic units. This study evaluates whether LLMs can identify meaningful revision unit boundaries in structured revision operation data and whether they provide value beyond simple non-LLM baselines. Using 113 matched draft--revision pairs from undergraduate writing, expert annotation yielded 4,344 candidate boundaries. We compared zero- and few-shot GPT-5.5 and base Qwen3-32B, parameter-efficient fine-tuning of Qwen3-32B, and majority and proximity-based baselines. Despite receiving revision context and task instructions, no prompted LLM condition outperformed the proximity heuristic (macro-F1 = .825). In contrast, fine-tuned Qwen3-32B using the two context representation achieved the highest macro-F1 (.859), identifying more same-unit relationships while maintaining precision comparable to the heuristic. Deterministic post-processing substantially improved the prompted models but added little benefit to the strongest fine-tuned model. These findings suggest that LLMs can support revision boundary judgment when task-adapted, but general purpose prompting alone may not outperform transparent structural heuristics.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.33720 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:02.725Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).