Metraipolis › archive › dc_fc6abe84ee57

Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models — arXiv:2609.05779

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.05779
License (as archived): CC0-1.0
Archived: 2026-10-16T14:08:58.757Z
SHA-256 of the archived text: a2174fc67d88478e1d5664a3c2850f7e4505ca45271180390aae7531cf3faf54
Stable URL of this page: https://metraipolis.com/documento/dc_fc6abe84ee57

How to cite

Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models — arXiv:2609.05779 — arXiv (en), https://arxiv.org/abs/2609.05779 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fc6abe84ee57 (Metraipolis archive dc_fc6abe84ee57, sha256 a2174fc67d88478e1d5664a3c2850f7e4505ca45271180390aae7531cf3faf54)

BibTeX

@misc{dcfc6abe84ee57,
  title = {Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models — arXiv:2609.05779},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.05779},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fc6abe84ee57; sha256 a2174fc67d88478e1d5664a3c2850f7e4505ca45271180390aae7531cf3faf54},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05779. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models used for code editing can be trained and deployed in at least two output regimes: direct generation, where the model emits the entire modified file in one shot, and iterative diff-based generation ("steps"), where the model emits a sequence of localized search/replace edits applied one at a time until it signals completion or a step budget is exhausted. The diff-based regime is attractive because it mirrors how developers edit code and should require far fewer generated tokens per turn. We train two code models - a 100M-parameter model trained from scratch (Rainbow-Pony-100M) and a fine-tuned Qwen2.5-Coder-0.5B - in both regimes on a shared Flutter/Dart dataset, and evaluate all four resulting models on a held-out set of approx 1,790 tasks per model. Direct generation substantially outperforms diff-based generation on every metric we measure - compilation/static-analysis pass rate, bits-per-byte, character-level similarity to the reference, and blinded LLM-judge ratings of goal fulfillment, correctness, and code quality - and the gap persists after controlling for task difficulty via a matched-ID comparison and when restricting to code that compiles on both sides. We then identify a single, architecture-independent mechanism behind the conditions where diff-based generation does win: it is competitive on short, spatially localized edits, and its category-level wins concentrate in exactly the two task categories - refactoring and error-handling/edge-case fixes - with the lowest mean edit-step count in our dataset. We term this task locality and discuss its implications for when an edit-based training regime is and is not the right choice for a code-editing model.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-09 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.05779 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:58:36.345Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).