Metraipolis › archive › dc_40fecfea4ad9
Stepwise-Querying: Distilling Reasoning without Access to Teacher Chain-of-Thought — arXiv:2610.41769
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41769
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T07:45:31.413Z
- SHA-256 of the archived text
0962830d0fb6e52f77a4928a129114de44e3d5631d75fd94213bd7cbd8eb3cb1- Stable URL of this page
https://metraipolis.com/documento/dc_40fecfea4ad9
How to cite
Stepwise-Querying: Distilling Reasoning without Access to Teacher Chain-of-Thought — arXiv:2610.41769 — arXiv (en), https://arxiv.org/abs/2610.41769 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_40fecfea4ad9 (Metraipolis archive dc_40fecfea4ad9, sha256 0962830d0fb6e52f77a4928a129114de44e3d5631d75fd94213bd7cbd8eb3cb1)
BibTeX
@misc{dc40fecfea4ad9,
title = {Stepwise-Querying: Distilling Reasoning without Access to Teacher Chain-of-Thought — arXiv:2610.41769},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41769},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_40fecfea4ad9; sha256 0962830d0fb6e52f77a4928a129114de44e3d5631d75fd94213bd7cbd8eb3cb1},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41769. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reasoning distillation is widely used to transfer the reasoning capabilities of large language models to smaller students through teacher-generated supervision. In the black-box setting, however, teachers may expose only final answers and brief solutions, leaving intermediate calculations and validity conditions unavailable and thus limiting the reasoning supervision available for distillation. We propose Stepwise Querying, which elicits step-specific explanations without requiring access to the teacher's internal chain-of-thought (CoT) traces. Independent queries to the same teacher request intermediate derivations, validity checks, and corrections for individual steps, the assembled solutions are then used for standard supervised fine-tuning. Across two teachers, two students, and four mathematics and code benchmarks, our method achieves the highest score in 14 of 16 comparisons with three baselines. We further introduce Stepwise Explanation Information Gain (SEIG), a diagnostic based on the log-ratio of a frozen student's probabilities of correct step completion under stepwise and global explanations. Analyses on GSM8K show that incorrect-answer trajectories can remain useful when they preserve valid intermediate reasoning, while further decomposition yields limited additional gains. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2610.41769 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T07:39:52.179Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).