Metraipolis › archive › dc_95ec8399256d

TSBPO: Tree-Structured Branching for Policy Optimization in Interactive Language Agents — arXiv:2610.41697

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41697
License (as archived)
CC0-1.0
Archived
2026-11-03T07:21:57.878Z
SHA-256 of the archived text
ae5ab1504b869ade09d8d491feabb7fef914a242656be33a98f1fa2f392fbbad
Stable URL of this page
https://metraipolis.com/documento/dc_95ec8399256d

How to cite

TSBPO: Tree-Structured Branching for Policy Optimization in Interactive Language Agents — arXiv:2610.41697 — arXiv (en), https://arxiv.org/abs/2610.41697 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_95ec8399256d (Metraipolis archive dc_95ec8399256d, sha256 ae5ab1504b869ade09d8d491feabb7fef914a242656be33a98f1fa2f392fbbad)
BibTeX
@misc{dc95ec8399256d,
  title = {TSBPO: Tree-Structured Branching for Policy Optimization in Interactive Language Agents — arXiv:2610.41697},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41697},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_95ec8399256d; sha256 ae5ab1504b869ade09d8d491feabb7fef914a242656be33a98f1fa2f392fbbad},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41697. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Training interactive language agents from terminal outcomes poses three coupled problems: which task and state continuations to sample, how to assign delayed feedback to decisions, and how to aggregate token losses for multi-token actions. We present TSBPO, a tree-structured sampling-to-update paradigm that addresses these problems within one training loop. A likelihood-fitted Root/Branch allocator uses retained task and state evidence to target continuations likely to produce contrasting terminal outcomes under fixed rollout budgets. On the resulting replayed physical trees, trajectory advantages compare immediate sibling segments, while observation-group step advantages compare outcomes at matched task/observation states. The actor then sums clipped token surrogates over each complete response and normalizes across active turns, avoiding response-length averaging in the policy-gradient term. With a Qwen2.5-1.5B actor, fixed checkpoints obtain 97.79% seen and 96.64% unseen stochastic success on ALFWorld and an 81.07% per-episode success rate over six stochastic episodes for each of the 500 WebShop test goals. A WebShop step-term ablation yields 75.93% at the same update-100 evaluation; an ALFWorld aggregation comparison shows higher observed command validity with complete-turn weighting. These results support the evaluated training configuration on two interactive benchmarks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.41697 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:19:48.837Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).