Metraipolis › archive › dc_961709f645aa

FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision — arXiv:2608.24350

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.24350
License (as archived)
CC0-1.0
Archived
2026-10-18T06:18:05.601Z
SHA-256 of the archived text
75041f4e83706a086a19dbe0468ac11d64447d6cceaa5bee81a33fd3f818278e
Stable URL of this page
https://metraipolis.com/documento/dc_961709f645aa

How to cite

FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision — arXiv:2608.24350 — arXiv (en), https://arxiv.org/abs/2608.24350 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_961709f645aa (Metraipolis archive dc_961709f645aa, sha256 75041f4e83706a086a19dbe0468ac11d64447d6cceaa5bee81a33fd3f818278e)
BibTeX
@misc{dc961709f645aa,
  title = {FARCA: Fact-Aligned Reliability-Aware Credit Assignment for Reinforcement Learning with Factual Supervision — arXiv:2608.24350},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.24350},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_961709f645aa; sha256 75041f4e83706a086a19dbe0468ac11d64447d6cceaa5bee81a33fd3f818278e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.24350. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

To reduce the hallucination risk caused by outcome-driven rewards in large language models trained through reinforcement learning with verifiable rewards, existing mitigation approaches introduce process-level factual supervision. However, due to coarse-grained aggregation of factual signals and the lack of reliability assessment for these signals, they create a mismatch between fact verification and policy updates. We term this noisy factual credit assignment and decompose it into two aspects: credit localization ambiguity and credit reliability ambiguity. To address these issues, we propose FARCA (Fact-Aligned Reliability-Aware Credit Assignment), a policy optimization framework that transforms factual supervision into localized, reliability-weighted token-level training signals. FARCA achieves fine-grained credit localization by aligning the granularity of fact verification with that of policy updates. It further introduces counterfactual evidence attribution, which uses the dependence of a factual judgment on key evidence as an empirical proxy for verification reliability to compute reliability weights. These weights modulate factual rewards and local policy advantages, reducing the influence of potentially unreliable signals on policy optimization. Experiments across different models and multiple factual reasoning benchmarks show that FARCA significantly improves model factuality while preserving general reasoning capabilities.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-26 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.24350 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T06:17:29.366Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-26?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).