Metraipolis › archive › dc_ef6e95bee393

CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity — arXiv:2608.07460

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.07460
License (as archived): CC0-1.0
Archived: 2026-10-19T09:47:03.462Z
SHA-256 of the archived text: 6cc58a49d1ff83dbe74b60a21cb3db5db39aa8339d7731c8bbef88b5112b7d52
Stable URL of this page: https://metraipolis.com/documento/dc_ef6e95bee393

How to cite

CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity — arXiv:2608.07460 — arXiv (en), https://arxiv.org/abs/2608.07460 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef6e95bee393 (Metraipolis archive dc_ef6e95bee393, sha256 6cc58a49d1ff83dbe74b60a21cb3db5db39aa8339d7731c8bbef88b5112b7d52)

BibTeX

@misc{dcef6e95bee393,
  title = {CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity — arXiv:2608.07460},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.07460},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef6e95bee393; sha256 6cc58a49d1ff83dbe74b60a21cb3db5db39aa8339d7731c8bbef88b5112b7d52},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07460. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

While post-training improves the capabilities of large language models (LLMs), it generally lowers their output diversity and creativity, negatively impacting tasks that explicitly require creativity (e.g., story generation) as well as those that require it implicitly, e.g., reinforcement learning (RL). We instead propose CreativeInstruct, a scalable instruction-tuning method that teaches LLMs to balance creative, base-model-like generations with the quality of post-trained models, by learning to inject special [StartCreativity] spans that bias generation toward creativity. Furthermore, we introduce a structural diversity metric based on graph edit distance, which captures narrative level variation missed by purely lexical and semantic metrics. On narrative generation, CreativeInstruct matches or exceeds the diversity of both multi-model baselines and distilled variants of their outputs, without sacrificing quality or requiring multiple models at inference time. These results are mirrored in our human evaluation, where we find that annotators rate CreativeInstruct generations as more creative than the post-trained LLMs' generations in 70.3% of cases. We also show the benefits of creative models as a substrate for RL: GRPO applied to a CreativeInstruct checkpoint improves by ~4% on AMC and ~5% points on MATH over the same training applied to the post-trained checkpoint.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-10 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.07460 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:46:37.251Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).