Metraipolis › archive › dc_e2c57b43d4bb
A Pinch of SFT, A Dash of RL: When Reinforcement Learning Helps Long-Horizon Advertising Agents — arXiv:2609.22194
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.22194
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T07:06:12.868Z
- SHA-256 of the archived text
7a6e5f5974580a9f6a37135765caf800c8dfd51dcd890a171e9f43b24b0b73ca- Stable URL of this page
https://metraipolis.com/documento/dc_e2c57b43d4bb
How to cite
A Pinch of SFT, A Dash of RL: When Reinforcement Learning Helps Long-Horizon Advertising Agents — arXiv:2609.22194 — arXiv (en), https://arxiv.org/abs/2609.22194 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e2c57b43d4bb (Metraipolis archive dc_e2c57b43d4bb, sha256 7a6e5f5974580a9f6a37135765caf800c8dfd51dcd890a171e9f43b24b0b73ca)
BibTeX
@misc{dce2c57b43d4bb,
title = {A Pinch of SFT, A Dash of RL: When Reinforcement Learning Helps Long-Horizon Advertising Agents — arXiv:2609.22194},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.22194},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e2c57b43d4bb; sha256 7a6e5f5974580a9f6a37135765caf800c8dfd51dcd890a171e9f43b24b0b73ca},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22194. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Enterprise analytics agents solve long-horizon tool-use problems over distributed business data, requiring retrieval, reasoning, API calls, code execution, and adaptation to intermediate observations. Supervised fine-tuning (SFT) calibrates tool syntax and teacher-supported behavior, whereas reinforcement learning (RL) can explore reward-supported behaviors beyond demonstrations; applied uniformly, however, RL can perturb already-calibrated skills. We study how to balance SFT and RL under production-mirroring beta APIs. We observe that, in our controlled experiment, checkpoint trajectories retrospectively separated into three regimes: Imitation, where SFT captured reliable teacher behavior; Lift, where both stages helped; and Discovery, where useful reward-observable behavior lay outside reliable teacher support. We leverage this prospectively, using teacher support and reward-observable headroom to route features to SFT only, SFT then RL, increased RL allocation, or further environment development. Across 18 subsequent feature-specific experiments, the diagnostic predicted 15/18 observed trajectories. On GPT-OSS 120B, targeted SFT then RL produced positive point estimates on 7/8 advertiser skills relative to a frontier Control; five positive gains had paired 95% confidence intervals excluding zero, while one skill had a confidence-supported regression. The largest gain was non-disclosure (+11.27 points; 95% CI [+9.72, +12.82]). A separate SME audit surfaced that targeted RL reduces standard leakage from 11.8% to 2.9% and adversarial leakage from 22.9% to 6.8% relative to SFT while preserving actionability (86.2% to 85.7%). In a matched uniform-versus-targeted comparison with shared rewards and optimization, targeted RL improved the seven-skill mean delta from +1.62 to +3.57 while using 43% less incremental RL compute. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA). Pagina del lavoro: https://arxiv.org/abs/2609.22194 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T06:58:09.155Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).