Metraipolis › archive › dc_e6ef96b43dad

SemComp-Bench: Benchmarking Semantic Task Completion in Video Generation — arXiv:2608.17426

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17426
License (as archived)
CC0-1.0
Archived
2026-10-18T14:33:31.655Z
SHA-256 of the archived text
4147125f8169c958ef83f33c9e4ffbdd2bc4f9c428a66389db6b753903f5a9ab
Stable URL of this page
https://metraipolis.com/documento/dc_e6ef96b43dad

How to cite

SemComp-Bench: Benchmarking Semantic Task Completion in Video Generation — arXiv:2608.17426 — arXiv (en), https://arxiv.org/abs/2608.17426 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e6ef96b43dad (Metraipolis archive dc_e6ef96b43dad, sha256 4147125f8169c958ef83f33c9e4ffbdd2bc4f9c428a66389db6b753903f5a9ab)
BibTeX
@misc{dce6ef96b43dad,
  title = {SemComp-Bench: Benchmarking Semantic Task Completion in Video Generation — arXiv:2608.17426},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17426},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e6ef96b43dad; sha256 4147125f8169c958ef83f33c9e4ffbdd2bc4f9c428a66389db6b753903f5a9ab},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17426. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We introduce Semantic Task Completion Video Generation, an outcome-oriented video generation task. Under this formulation, success requires both achievement of the intended outcome and semantic grounding. Semantic grounding characterizes the correspondence between the reference image and the generated outcome in terms of high-level semantics relevant to the task. Evaluation focuses on the generated outcome and requires neither the presentation of a complete sequence of intermediate task steps nor conventional appearance consistency with the reference image. To support systematic evaluation, we construct SemComp-Data, an evaluation dataset covering six domains. Each instance comprises a reference image, a detailed instruction, a brief instruction, and an outcome-centric video clip. A scalable four-stage curation pipeline converts raw videos into standardized SemComp-Data instances. We further introduce SemComp-Bench, an evaluation protocol that uses a vision-language model (VLM) to answer structured binary questions. SemComp-Bench reports the OA Score and the GR Score for Outcome Achievement and Generation Reliability, respectively. Experiments on representative video generation models show that achieving intended outcomes while maintaining task-relevant semantic grounding in reference images remains challenging.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.17426 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:27:54.428Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).