Metraipolis › archive › dc_db028fd423ab

HoosierHelp: Benchmarking LLM Agents for Social Service Navigation — arXiv:2608.09946

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.09946
License (as archived): CC0-1.0
Archived: 2026-10-19T06:08:46.916Z
SHA-256 of the archived text: 4ee49af0d800a34ee27cd8887aae6a51c09fa5b02a02550ea8dcba4169484f6a
Stable URL of this page: https://metraipolis.com/documento/dc_db028fd423ab

How to cite

HoosierHelp: Benchmarking LLM Agents for Social Service Navigation — arXiv:2608.09946 — arXiv (en), https://arxiv.org/abs/2608.09946 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_db028fd423ab (Metraipolis archive dc_db028fd423ab, sha256 4ee49af0d800a34ee27cd8887aae6a51c09fa5b02a02550ea8dcba4169484f6a)

BibTeX

@misc{dcdb028fd423ab,
  title = {HoosierHelp: Benchmarking LLM Agents for Social Service Navigation — arXiv:2608.09946},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09946},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_db028fd423ab; sha256 4ee49af0d800a34ee27cd8887aae6a51c09fa5b02a02550ea8dcba4169484f6a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09946. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Social service navigation requires connecting help-seeking individuals to resources that satisfy their needs and specific constraints. Although LLM agents offer a promising interface for conversational resource navigation, existing benchmarks do not capture the interaction complexity and constraint-grounding demands of this setting. We introduce HoosierHelp, an interactive benchmark grounded in 3,971 Indiana public social service resources. Agents interact with simulated users, issue structured resource-search calls, handle non-ideal interactions, and select the final resources returned by the tool. HoosierHelp enhances the realism of simulated users by varying their need structure, constraint satisfiability, and behavior patterns, including impatience, rambling, unsupported requests, and self-contradiction. Experiments on 240 samples across seven LLMs show that current LLM agents remain substantially unreliable for social service navigation. Performance drops sharply on fallback-required and self-contradictory conversations, highlighting the need for agents that are more robust to complex and non-ideal user interactions.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.AI).
Categorie arXiv: Human-Computer Interaction (cs.HC) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.09946 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:57:28.319Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).