Metraipolis › archive › dc_c9ab1efdbcce

Code Consistency Preference Optimization Verification for Language Model Alignment — arXiv:2609.19002

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.19002
License (as archived)
CC0-1.0
Archived
2026-10-15T13:50:03.277Z
SHA-256 of the archived text
ac471e331ebf6fd13bb255a47a915e97ae7fb638409e79b1e699af451402754a
Stable URL of this page
https://metraipolis.com/documento/dc_c9ab1efdbcce

How to cite

Code Consistency Preference Optimization Verification for Language Model Alignment — arXiv:2609.19002 — arXiv (en), https://arxiv.org/abs/2609.19002 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c9ab1efdbcce (Metraipolis archive dc_c9ab1efdbcce, sha256 ac471e331ebf6fd13bb255a47a915e97ae7fb638409e79b1e699af451402754a)
BibTeX
@misc{dcc9ab1efdbcce,
  title = {Code Consistency Preference Optimization Verification for Language Model Alignment — arXiv:2609.19002},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19002},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c9ab1efdbcce; sha256 ac471e331ebf6fd13bb255a47a915e97ae7fb638409e79b1e699af451402754a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19002. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Execution-based verification enhances large language models' mathematical reasoning through computational soundness and dependency-aware filtering. However, prior preference optimization methods relying on Bradley-Terry reward models fail to capture the logical dependencies and execution consistency needed for scientific tasks. We propose a method that generates computationally sound solutions with dependency graphs for execution-consistent preference optimization. We first build a scientific reasoning dataset using UltraFeedback prompts, model generations, verification, and consistency results. Then we extract reasoning step expressions, prerequisites, and derivability relationships to construct dependency graphs and compute execution consistency scores. These scores are appended to each step, creating paired training data. Fine-tuning Llama-3-8B and DeepSeekMath-7B yields significant gains: +17.0% on MATH and +15.1% on GSM8K. Extending our Scientific Feasibility Control framework achieves 50.1% accuracy on PhyX multimodal physics reasoning, surpassing DeepSeek-R1 (49.8%) and OpenAI o3-mini (48.2%), with 91.7% scientific validity coverage at alpha=0.10 and 73% fewer scientific law violations, resulting in the CCPO model family.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CL).
Categorie arXiv: Software Engineering (cs.SE) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.19002 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:48:24.444Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).