Metraipolis › archive › dc_4b2bec617463
How Much Does It Cost to Answer My Question? Benchmarking Cloud VLM-based VQA Systems — arXiv:2608.07861
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.07861
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T09:15:40.473Z
- SHA-256 of the archived text
c480efe7e30f735ca16b375fe6f01c5c5bc299ef9710a4140f420b0a23c1d9df- Stable URL of this page
https://metraipolis.com/documento/dc_4b2bec617463
How to cite
How Much Does It Cost to Answer My Question? Benchmarking Cloud VLM-based VQA Systems — arXiv:2608.07861 — arXiv (en), https://arxiv.org/abs/2608.07861 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4b2bec617463 (Metraipolis archive dc_4b2bec617463, sha256 c480efe7e30f735ca16b375fe6f01c5c5bc299ef9710a4140f420b0a23c1d9df)
BibTeX
@misc{dc4b2bec617463,
title = {How Much Does It Cost to Answer My Question? Benchmarking Cloud VLM-based VQA Systems — arXiv:2608.07861},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.07861},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4b2bec617463; sha256 c480efe7e30f735ca16b375fe6f01c5c5bc299ef9710a4140f420b0a23c1d9df},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07861. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Vision-language models (VLMs) are becoming a practical backend for mobile visual question answering (VQA) systems, enabling smartphones and smart glasses to answer users' questions about the physical world. Since modern VLMs remain difficult to run on mobile and edge devices, VQA systems increasingly offload inference to cloud-based VLMs. This gives mobile devices access to stronger computation, but it also makes visual input preparation a key system variable: how the image is prepared before offloading affects not only answer quality but also payload size, token cost, and system latency. Proprietary APIs expose little control over model internals or serving behavior, leaving client-side preprocessing as the main practical optimization space for downstream developers. Many such techniques have been proposed for visual offloading, yet their cost-quality impact on commercial cloud VLMs has never been studied. To fill this gap, we present VQABench, the first systematic benchmark that treats client-side input preprocessing as a controlled variable for cloud-VLM-based VQA. We evaluate 12 preprocessing techniques across three VQA datasets and four commercial VLMs from three providers, totaling 95,168 API calls. Our results show that preprocessing is not universally beneficial: its effectiveness depends on the target model, API paradigm, provider token-accounting rule, and task formulation. A poorly selected preprocessing strategy can increase deployment cost or latency while degrading answer accuracy. Overall, our benchmark clarifies when preprocessing helps, when it fails, and why, providing insights to guide future research and real-world deployment of VQA systems. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.MM). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Human-Computer Interaction (cs.HC); Information Retrieval (cs.IR); Multimedia (cs.MM). Pagina del lavoro: https://arxiv.org/abs/2608.07861 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T09:13:27.075Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).