Metraipolis › archive › dc_45fa21dc7a72

NutriVisRAG: Multimodal Exemplar Grounding for Nutrient Estimation with Vision-Language Models — arXiv:2611.08054

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.08054
License (as archived)
CC0-1.0
Archived
2026-11-06T14:42:48.271Z
SHA-256 of the archived text
5473eb454550f916df5c0619bc2a4c9b96acaaaf394ddcf565a915fb24cf6ae4
Stable URL of this page
https://metraipolis.com/documento/dc_45fa21dc7a72

How to cite

NutriVisRAG: Multimodal Exemplar Grounding for Nutrient Estimation with Vision-Language Models — arXiv:2611.08054 — arXiv (en), https://arxiv.org/abs/2611.08054 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_45fa21dc7a72 (Metraipolis archive dc_45fa21dc7a72, sha256 5473eb454550f916df5c0619bc2a4c9b96acaaaf394ddcf565a915fb24cf6ae4)
BibTeX
@misc{dc45fa21dc7a72,
  title = {NutriVisRAG: Multimodal Exemplar Grounding for Nutrient Estimation with Vision-Language Models — arXiv:2611.08054},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.08054},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_45fa21dc7a72; sha256 5473eb454550f916df5c0619bc2a4c9b96acaaaf394ddcf565a915fb24cf6ae4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08054. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Diet is a major determinant of health, yet accurate dietary assessment from free-living meal images remains challenging. Existing retrieval-based Vision-Language Model (VLM) approaches to nutrient estimation typically rely on textual food representations, while, to our knowledge, direct grounding on visually similar annotated meals has not been investigated. We present NutriVisRAG, a retrieval-augmented generation framework that uses a vision encoder to retrieve similar meals with known nutritional annotations and provide them as multimodal in-context exemplars for VLM nutrient estimation at inference time, without task-specific fine-tuning. We evaluate NutriVisRAG across four datasets, four VLMs, three retrieval encoders, and two prompting strategies, including free-living meals from adults with insulin-treated diabetes, where carbohydrate estimates are compared against two dietitians' annotations. Across all 96 configurations (4 datasets x 4 VLMs x 3 retrieval encoders x 2 prompting strategies), NutriVisRAG reduced the primary estimation error in 56, with improvements more frequent under whole-meal (35/48) than per-item prompting (21/48). Across the evaluated configurations, the lowest NutriVisRAG error was below the best Vanilla error in 13 of 16 VLM--dataset pairs. For GPT-5.4-mini, under whole-meal prompting, the best-performing retrieval encoder for each dataset reduced the primary error on all four datasets relative to Vanilla, with reductions of 1.7--35.7%. Overall, our results show that grounding in multimodal exemplars comprising images and respective annotations can improve nutrient estimation at inference time without task-specific fine-tuning, although the benefits remain configuration-dependent.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.08054 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.696Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).