Metraipolis › archive › dc_0393daeec37e

Online Video Agent Harness for Long Video Understanding — arXiv:2609.12818

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.12818
License (as archived)
CC0-1.0
Archived
2026-10-16T03:03:37.536Z
SHA-256 of the archived text
7f3c1a8aa72b1a18167c9f0c62ea7e598df3e14420eed81314cdb7eab470eaa5
Stable URL of this page
https://metraipolis.com/documento/dc_0393daeec37e

How to cite

Online Video Agent Harness for Long Video Understanding — arXiv:2609.12818 — arXiv (en), https://arxiv.org/abs/2609.12818 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0393daeec37e (Metraipolis archive dc_0393daeec37e, sha256 7f3c1a8aa72b1a18167c9f0c62ea7e598df3e14420eed81314cdb7eab470eaa5)
BibTeX
@misc{dc0393daeec37e,
  title = {Online Video Agent Harness for Long Video Understanding — arXiv:2609.12818},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.12818},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0393daeec37e; sha256 7f3c1a8aa72b1a18167c9f0c62ea7e598df3e14420eed81314cdb7eab470eaa5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.12818. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long video understanding often behaves like a visual needle-in-a-haystack problem: query-relevant evidence is sparsely distributed across long temporal spans, while packing dense frames into a single VLM context incurs \textit{context rot} and high cost. Existing video agents often rely on query-agnostic offline preprocessing or ad hoc tool sets, which can miss query-specific details and waste computation. In this work, we present VideoXAgent, a purely online video-agent harness for long video understanding that starts from the given video file and user query, plans and decomposes the task, invokes specialized expert tools on demand, and aggregates multimodal evidence to produce a final answer while resolving conflicts among observations. To support this on-demand invocation, we design a suite of heterogeneous expert tools guided by a data-driven taxonomy of atomic capabilities, spanning scripts, VLMs, and domain models (e.g., detection, OCR, ASR, face recognition). The harness further enforces objective evidence prompting and budget-aware control to curb hallucination and non-termination. Across Video-MME-Long, LongVideoBench-Long, LVBench, and MINERVA, VideoXAgent is competitive with frontier LMMs and video agents under a smaller context footprint---about 50k tokens of agent context per sample, even on hour-long videos. In particular, on complex video-reasoning benchmarks such as MINERVA, it matches this level while using only about 15\% of the context of a 1,024-frame dense-packing baseline. Notably, the harness remains effective with a visually weak or even text-only orchestrator, suggesting that strong long-video understanding can emerge from progressive agentic evidence seeking rather than from packing the full video into a single context. Project page: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.12818 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T02:51:27.944Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).