Metraipolis › archive › dc_efa1fa4a5b94
Content Based Video Narration of Gameplay with Vision Language Models — arXiv:2608.14016
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14016
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T02:21:15.224Z
- SHA-256 of the archived text
034fc8127dd2db013f4533e58397ec98f43ec13b6d896cfa75270d574c4e5b30- Stable URL of this page
https://metraipolis.com/documento/dc_efa1fa4a5b94
How to cite
Content Based Video Narration of Gameplay with Vision Language Models — arXiv:2608.14016 — arXiv (en), https://arxiv.org/abs/2608.14016 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_efa1fa4a5b94 (Metraipolis archive dc_efa1fa4a5b94, sha256 034fc8127dd2db013f4533e58397ec98f43ec13b6d896cfa75270d574c4e5b30)
BibTeX
@misc{dcefa1fa4a5b94,
title = {Content Based Video Narration of Gameplay with Vision Language Models — arXiv:2608.14016},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14016},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_efa1fa4a5b94; sha256 034fc8127dd2db013f4533e58397ec98f43ec13b6d896cfa75270d574c4e5b30},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14016. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Live game commentary is scarce: it exists for professional esports broadcasts and almost nowhere else. We present a content-based video narration system that produces spoken, esports-style commentary for arbitrary gameplay recordings using a general-purpose vision-language model (VLM) and a text-to-speech back end, with no game-specific instrumentation, no engine telemetry, and no task-specific training. Three mechanisms carry the system. Temporal mosaic packing arranges nine uniformly sampled frames into a single 3x3 image, letting an image-native VLM reason about motion while consuming one image payload per segment instead of nine. Context-conditioned prompting replays the K most recent narrations as assistant-role history, suppressing the repetition that dominates per-segment captioning of static scenes. Duration-conditioned generation and elastic alignment constrain narration length in the prompt, then time-scale or symmetrically pad the synthesized audio so each utterance fills its segment slot exactly, giving frame-accurate muxing without a forced aligner. The implementation supports either cloud TTS or a 6-bit quantized 4B-parameter on-device TTS model on Apple silicon, making the speech stage fully local. We report a qualitative case study on real-time strategy footage, a cost model showing the mosaic reduces per-minute image payloads by 9x, and a candid account of observed failure modes - hallucinated game state, resolution loss from mosaicking, and prosody artifacts from time-scaling. We release the system as a reproducible baseline, with an evaluation protocol for the quantitative study a full version will report. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Graphics (cs.GR). Pagina del lavoro: https://arxiv.org/abs/2608.14016 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T02:02:02.892Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).