Metraipolis › archive › dc_83be3ebeae6f

Scaling Qualitative Analysis with Large Language Models: Development and Validation of the Smart Coder Framework for Human-LLM Collaborative Coding — arXiv:2610.33205

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.33205
License (as archived)
CC0-1.0
Archived
2026-10-27T07:56:29.339Z
SHA-256 of the archived text
253c60cdfcd4af346cf2e7a5175005fa77590d7bf634fdd7768633dc092c6fec
Stable URL of this page
https://metraipolis.com/documento/dc_83be3ebeae6f

How to cite

Scaling Qualitative Analysis with Large Language Models: Development and Validation of the Smart Coder Framework for Human-LLM Collaborative Coding — arXiv:2610.33205 — arXiv (en), https://arxiv.org/abs/2610.33205 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_83be3ebeae6f (Metraipolis archive dc_83be3ebeae6f, sha256 253c60cdfcd4af346cf2e7a5175005fa77590d7bf634fdd7768633dc092c6fec)
BibTeX
@misc{dc83be3ebeae6f,
  title = {Scaling Qualitative Analysis with Large Language Models: Development and Validation of the Smart Coder Framework for Human-LLM Collaborative Coding — arXiv:2610.33205},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.33205},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_83be3ebeae6f; sha256 253c60cdfcd4af346cf2e7a5175005fa77590d7bf634fdd7768633dc092c6fec},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.33205. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Current uses of large language models (LLMs) for qualitative coding often rely on ad hoc prompting with limited guidance for reliability and human oversight. To address these limitations, we propose the Smart Coder Framework, a three-phase approach structuring human-LLM collaborative coding through Instruction, Prompt Strategy, and Reliability Check. We evaluated the framework using 72 middle-school mathematics teacher interview transcripts and a human-consensus benchmark. A factorial design compared instructed versus uninstructed prompts, few-shot versus few-shot plus chain-of-thought prompting, and single-pass versus self-consistency coding. Twelve transcripts were used for prompt development and calibration, and 60 for the formal evaluation. In the results, instruction consistently improved agreement. Chain-of-thought further improved agreement under adequate instruction, and self-consistency produced smaller, theme-dependent gains but helped identify unstable decisions for human review. Performance also varied by transcript characteristics. Overall, the framework systematically structures and monitors LLM-assisted qualitative coding while preserving human authority over coding decisions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.33205 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:30.622Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).