Metraipolis › archive › dc_f0e101f417d8

ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts — arXiv:2609.18844

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.18844
License (as archived)
CC0-1.0
Archived
2026-10-15T13:49:35.957Z
SHA-256 of the archived text
cc40d4362a1b7789f965586d222d4e65c6e428699df90ed45d58fc63aa231e55
Stable URL of this page
https://metraipolis.com/documento/dc_f0e101f417d8

How to cite

ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts — arXiv:2609.18844 — arXiv (en), https://arxiv.org/abs/2609.18844 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f0e101f417d8 (Metraipolis archive dc_f0e101f417d8, sha256 cc40d4362a1b7789f965586d222d4e65c6e428699df90ed45d58fc63aa231e55)
BibTeX
@misc{dcf0e101f417d8,
  title = {ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts — arXiv:2609.18844},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.18844},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f0e101f417d8; sha256 cc40d4362a1b7789f965586d222d4e65c6e428699df90ed45d58fc63aa231e55},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18844. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal coding agents are expected to turn visual inputs into usable artifacts, and they act through a harness, the layer of tools, context management, and execution environment around the model. Existing evaluations often isolate short tool calls, API traces, or screenshot resemblance, and a low score under these proxies cannot say whether the model saw poorly, planned poorly, or was failed by its harness. We study scientific overview figure reconstruction, an agent task in which a source image must become an editable PowerPoint slide that preserves text, topology, layout, and native document structure. We introduce ReFigBench, a benchmark and evaluation framework built on 1,000 real overview figures retrieved from arXiv papers with full provenance. Coding agents from four model families reconstruct every figure under two workflows, direct code generation and a specialized PPTX workflow, and the strongest model runs inside two commercial harnesses, yielding ten configurations. Evaluation combines deterministic artifact checks, repeated automated scoring by judges from two model families, and blinded human comparisons. Perception remains a bottleneck that iterative rendering only partly repays. Whether workflow effort converts into quality depends on the model together with its harness, since the same model gains from the specialized workflow inside one harness and loses inside the other, and the harness shifts scores even under an identical direct prompt. The specialized workflow erases native connectors in every configuration, human judges still prefer its renderings in most matchups, and even the strongest agent falls short of the rubric ceiling. These results expose the tension between fidelity and editability as the central challenge for practical multimodal document agents.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.18844 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:48:24.345Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).