Metraipolis › archive › dc_3f91a58bb40e

Can Large Language Model Agents Learn by Watching Tutorial Videos? — arXiv:2611.01460

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.01460
License (as archived)
CC0-1.0
Archived
2026-11-03T10:26:23.340Z
SHA-256 of the archived text
8fa8000bd1cd965e093e75fad09d5899dabb5a2529a946ede5c549be4cbd0a47
Stable URL of this page
https://metraipolis.com/documento/dc_3f91a58bb40e

How to cite

Can Large Language Model Agents Learn by Watching Tutorial Videos? — arXiv:2611.01460 — arXiv (en), https://arxiv.org/abs/2611.01460 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3f91a58bb40e (Metraipolis archive dc_3f91a58bb40e, sha256 8fa8000bd1cd965e093e75fad09d5899dabb5a2529a946ede5c549be4cbd0a47)
BibTeX
@misc{dc3f91a58bb40e,
  title = {Can Large Language Model Agents Learn by Watching Tutorial Videos? — arXiv:2611.01460},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.01460},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3f91a58bb40e; sha256 8fa8000bd1cd965e093e75fad09d5899dabb5a2529a946ede5c549be4cbd0a47},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.01460. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Humans continuously acquire knowledge from carefully crafted materials such as books and tutorial videos, carrying that understanding into future tasks. Millions of such instructional videos are uploaded online every day, yet LLM agents remain unable to exploit this ever-expanding wealth of human teaching. Their capabilities are frozen after training, while parameter updates at scale remain prohibitively expensive. This raises the central question: can LLM agents learn from this vast instructional resource without parameter updates? We introduce Tutorial Agentic Learning (TAL), a nonparametric continual-learning paradigm that consolidates tutorial videos into a persistent, structured tutorialbook memory through a two-stage adversarial process that harvests hard, video-grounded questions and uses them to refine the memory. At inference time, TAL retrieves through a Table-of-Contents (TOC) tree, tying retrieval directly to the model's reasoning rather than to embedding similarity. To study this setting, we introduce VidLearn, a multimodal QA benchmark built from 72 chess tutorial videos spanning 12 openings. On VidLearn, TAL outperforms all evaluated baselines and improves monotonically as more videos are consumed, achieving up to 113% and 91% relative gains over the corresponding vanilla MLLMs on the training and test splits, respectively. In actual gameplay, TAL reaches a stronger opening-phase position than its vanilla counterpart in 69% of head-to-head games. Evaluation in a second domain further demonstrates the generality of TAL. Together, these results highlight instructional videos as a scalable resource for parameter-update-free agent continual learning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.01460 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:19:50.077Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).