Metraipolis › archive › dc_df51549eaccb

FramingQA: Does the Question Shape the Answer? Measuring the Compositional Framing Effect — arXiv:2609.07448

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.07448
License (as archived): CC0-1.0
Archived: 2026-10-16T12:45:36.136Z
SHA-256 of the archived text: 9f920a3941d45e973f2769753df50aebffafe4a235eb64335f6cadb71359646b
Stable URL of this page: https://metraipolis.com/documento/dc_df51549eaccb

How to cite

FramingQA: Does the Question Shape the Answer? Measuring the Compositional Framing Effect — arXiv:2609.07448 — arXiv (en), https://arxiv.org/abs/2609.07448 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_df51549eaccb (Metraipolis archive dc_df51549eaccb, sha256 9f920a3941d45e973f2769753df50aebffafe4a235eb64335f6cadb71359646b)

BibTeX

@misc{dcdf51549eaccb,
  title = {FramingQA: Does the Question Shape the Answer? Measuring the Compositional Framing Effect — arXiv:2609.07448},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.07448},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_df51549eaccb; sha256 9f920a3941d45e973f2769753df50aebffafe4a235eb64335f6cadb71359646b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.07448. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We introduce FramingQA, a benchmark that measures the model sensitivity to question framing across law, medicine, finance, and robotic simulations. Large language models (LLMs) often change their responses to subtle rephrasings that align with an implied stance by users. This can leave users with advice tainted by how they happened to phrase a question rather than by the underlying facts, and the consequences are highly costly in high-stakes domains. Because in the realistic scenarios, both expert practitioners and non-expert users frequently ask LLMs questions containing incomplete or misleading assumptions, models are highly susceptible to those framings. To test this, we inject the framing bias across three nested levels: a framing-biased question phrasing (root), an injected framing-biased premise prepended to a neutral question (propositional), and a premise paired with a framing-biased question (global). Evaluating nine open models (3.8B-70B) across four families, we find that strong per-variant accuracy does not guarantee the robustness across differently phrased questions under the fixed factual information.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.07448 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:25.120Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).