Metraipolis › archive › dc_81d736ff9b3a
AutoCue: Multimodal LLM-Assisted Externalization of Implicit Inputs as Instructional Visual Cues in Screencast Tutorials — arXiv:2608.04910
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.04910
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T11:28:28.405Z
- SHA-256 of the archived text
2d0a0268833109ec5482b2f2339e2b3d3cb6c3881a4a6cb0720874d8fb4b7505- Stable URL of this page
https://metraipolis.com/documento/dc_81d736ff9b3a
How to cite
AutoCue: Multimodal LLM-Assisted Externalization of Implicit Inputs as Instructional Visual Cues in Screencast Tutorials — arXiv:2608.04910 — arXiv (en), https://arxiv.org/abs/2608.04910 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_81d736ff9b3a (Metraipolis archive dc_81d736ff9b3a, sha256 2d0a0268833109ec5482b2f2339e2b3d3cb6c3881a4a6cb0720874d8fb4b7505)
BibTeX
@misc{dc81d736ff9b3a,
title = {AutoCue: Multimodal LLM-Assisted Externalization of Implicit Inputs as Instructional Visual Cues in Screencast Tutorials — arXiv:2608.04910},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.04910},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_81d736ff9b3a; sha256 2d0a0268833109ec5482b2f2339e2b3d3cb6c3881a4a6cb0720874d8fb4b7505},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04910. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Tutorial videos are widely used for learning feature-rich software, yet following screencast tutorials often breaks down in practice. Through a survey and contextual inquiry, we found that learners frequently rewind or get stuck because critical input information, especially mouse actions and keyboard-modified operations, is often implicit or missing in tutorials without input metadata. To address this problem, we present AutoCue, a multimodal LLM-assisted, human-in-the-loop tutorial augmentation pipeline for externalizing implicit inputs as instructional visual cues. AutoCue integrates frame-to-frame visual changes, narration signals, and operation guidance from official software manuals to infer likely mouse and key-modifier actions, then produces aligned cue layers and editable artifacts for human refinement. Grounded in multimedia learning and cognitive load theory, we further develop a visual cue grammar for representing mouse, keyboard, and combined inputs in software-learning tutorials. We instantiate and evaluate AutoCue in Autodesk Maya, focusing automatic inference on selected UI-mediated interactions with observable visual or textual feedback while supporting more ambiguous state changes through editable authoring artifacts. In a between-subjects study with 24 participants, the AutoCue-augmented tutorial reduced task completion time and interaction breakdowns and showed improved learner-reported experience. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-06 in cs.HC). Categorie arXiv: Human-Computer Interaction (cs.HC). Pagina del lavoro: https://arxiv.org/abs/2608.04910 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T11:17:34.226Z dalla pagina https://arxiv.org/catchup/cs.HC/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).