Metraipolis › archive › dc_ff4e61c75acc

A Factorial Ablation of a Speech-to-SFT Pipeline: Differential Effects on Data Quality and Downstream Transfer — arXiv:2608.20394

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.20394
License (as archived): CC0-1.0
Archived: 2026-10-18T10:18:35.736Z
SHA-256 of the archived text: 5668c16d74d4514d733e06ce51cb11dc07f43fb9270d0eddf2138454c930d176
Stable URL of this page: https://metraipolis.com/documento/dc_ff4e61c75acc

How to cite

A Factorial Ablation of a Speech-to-SFT Pipeline: Differential Effects on Data Quality and Downstream Transfer — arXiv:2608.20394 — arXiv (en), https://arxiv.org/abs/2608.20394 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ff4e61c75acc (Metraipolis archive dc_ff4e61c75acc, sha256 5668c16d74d4514d733e06ce51cb11dc07f43fb9270d0eddf2138454c930d176)

BibTeX

@misc{dcff4e61c75acc,
  title = {A Factorial Ablation of a Speech-to-SFT Pipeline: Differential Effects on Data Quality and Downstream Transfer — arXiv:2608.20394},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20394},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ff4e61c75acc; sha256 5668c16d74d4514d733e06ce51cb11dc07f43fb9270d0eddf2138454c930d176},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20394. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Industry pipelines that turn speech into supervised fine-tuning (SFT) data via multi-stage refinement are increasingly adopted but, to our knowledge, have not been publicly ablated stage-by-stage, leaving each stage's marginal value unknown. We design a production-ready speech-to-SFT pipeline in which transcript refinement (Phase 0) and SFT data quality refinement (Phase 2) are independently toggleable, yielding a 2x2 factorial design. For each condition, we generate QA-form SFT data from Korean medical and finance conference recordings and fine-tune 9 models (5 LLM families, 2.4B-70B); we evaluate with four cross-provider LLM judges, a blind six-expert human evaluation, and 3 downstream MCQA benchmarks. Our central finding: under a fixed, standard SFT recipe, improvements in QA data quality do not transfer uniformly into downstream MCQA gains. 4-judge quality rises consistently, yet the cross-model mean MCQA gain is not significant; positive transfer concentrates on family-domain aligned pairs. This differential pattern is consistent with a format mismatch: Phase 2 shifts SFT-data composition toward explanatory items, while MCQA primarily probes factoid recall. All six human raters report higher full-pipeline quality, confirming the LLM-judge direction. An STT-engine swap to Whisper-medium confirms pipeline robustness. A non-hallucination audit shows the two frontier LLMs admit unknown on approximately 8% of QA on average; we release samples, prompts, code, and all SFT checkpoints.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-24 in cs.CL).
Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.20394 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:17:37.606Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).