Metraipolis › archive › dc_e5ab88017ed2
MathShikkha: A Controlled Study of Answer-Only and Chain-of-Thought Supervision for Bangla Mathematical Reasoning in Small Language Models — arXiv:2608.08503
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.08503
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T08:50:48.940Z
- SHA-256 of the archived text
cb3846a85ca2ff4d833dfc4485a7c380112cb90acade8cacc6f9c96e0335a4f3- Stable URL of this page
https://metraipolis.com/documento/dc_e5ab88017ed2
How to cite
MathShikkha: A Controlled Study of Answer-Only and Chain-of-Thought Supervision for Bangla Mathematical Reasoning in Small Language Models — arXiv:2608.08503 — arXiv (en), https://arxiv.org/abs/2608.08503 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e5ab88017ed2 (Metraipolis archive dc_e5ab88017ed2, sha256 cb3846a85ca2ff4d833dfc4485a7c380112cb90acade8cacc6f9c96e0335a4f3)
BibTeX
@misc{dce5ab88017ed2,
title = {MathShikkha: A Controlled Study of Answer-Only and Chain-of-Thought Supervision for Bangla Mathematical Reasoning in Small Language Models — arXiv:2608.08503},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.08503},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e5ab88017ed2; sha256 cb3846a85ca2ff4d833dfc4485a7c380112cb90acade8cacc6f9c96e0335a4f3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08503. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Mathematical reasoning remains challenging in low-resource languages such as Bangla. We study whether teacher-generated Bangla Chain-of-Thought (CoT) supervision provides benefits beyond ordinary supervised fine-tuning. We construct \textsc{MathShikkha}, a Bangla mathematical reasoning dataset with GPT-5.4-generated rationales, and fine-tune four 4B--7B student models under a matched protocol in which answer-only and CoT conditions share data splits, response-only loss masking, decoding, and scoring, differing only in the training target. In-domain, CoT provides no significant improvement over answer-only fine-tuning for three stronger backbones (paired bootstrap 95\% CIs include zero; exact McNemar $p \geq 0.17$), despite generating 15--52$\times$ more tokens, but significantly improves the weaker 4B model by 18.56 points ($p < 0.0001$). On the larger, contamination-audited BanglaMATH benchmark, this pattern reverses: CoT significantly outperforms answer-only supervision for all four models by 20.1--28.1 points (all $p < 0.0001$). Answer-only fine-tuning also reduces out-of-domain accuracy below the base model for three models, whereas CoT preserves or improves it for all four. A human study with two co-author annotators, external-expert adjudication, and Cohen's $\kappa = 0.76$--$1.00$ finds no significant CoT improvement over the base model on reasoning-content criteria; instead, its measurable effect is target-language adherence and producing inspectable reasoning. Overall, rationale supervision's value depends on backbone capability and distribution shift: in this setting, its main benefits are Bangla adherence, auditable reasoning, and out-of-domain robustness rather than improved in-domain reasoning validity.
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.08503 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.986Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).