Metraipolis › archive › dc_fac896e6bd7a

Pseudo2CodeQA: A Benchmark for LLM-Based Structured Algorithmic Reasoning in Code Generation — arXiv:2608.09068

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.09068
License (as archived)
CC0-1.0
Archived
2026-10-19T09:00:58.178Z
SHA-256 of the archived text
c5b3e438826c94af89f2a80e04ac74d4e0fc71caade4431aa7587e7a5a5dccd3
Stable URL of this page
https://metraipolis.com/documento/dc_fac896e6bd7a

How to cite

Pseudo2CodeQA: A Benchmark for LLM-Based Structured Algorithmic Reasoning in Code Generation — arXiv:2608.09068 — arXiv (en), https://arxiv.org/abs/2608.09068 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fac896e6bd7a (Metraipolis archive dc_fac896e6bd7a, sha256 c5b3e438826c94af89f2a80e04ac74d4e0fc71caade4431aa7587e7a5a5dccd3)
BibTeX
@misc{dcfac896e6bd7a,
  title = {Pseudo2CodeQA: A Benchmark for LLM-Based Structured Algorithmic Reasoning in Code Generation — arXiv:2608.09068},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09068},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fac896e6bd7a; sha256 c5b3e438826c94af89f2a80e04ac74d4e0fc71caade4431aa7587e7a5a5dccd3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09068. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) have achieved impressive performance in natural language-to-code generation; however, their ability to follow structured algorithmic reasoning remains insufficiently understood. We introduce Pseudo2Code, a benchmark designed to systematically evaluate the impact of structured pseudocode on code generation quality and algorithmic faithfulness. The benchmark consists of 300 manually validated real-world programming tasks spanning multiple domains and three difficulty levels (Easy, Medium, and Hard). Each task contains a problem description, structured pseudocode, reference implementation, and executable test suite. To ensure benchmark reliability, we adopt a dual-stage human validation protocol and release fully executable benchmark instances. Beyond the benchmark, we propose the Pseudo2Code Agentic Framework, a multi-stage pipeline that leverages pseudocode as an explicit intermediate reasoning representation for code generation. We evaluate both commercial and open-source language models using a rubric-based evaluation framework that measures correctness, completeness, relevance, clarity, reasoning quality, and pseudocode adherence, complemented by execution-based testing. Experimental results demonstrate that the proposed Pseudo2Code Agentic Pipeline consistently outperforms strong commercial and open-source baselines, achieving an overall score of 4.78 compared to 4.31 for the strongest baseline model. Furthermore, a human evaluation study involving 100 benchmark tasks shows strong agreement between human judgments and automated assessments. Our findings provide empirical evidence that structured pseudocode improves functional correctness, reasoning quality, and algorithmic faithfulness in code generation. We release Pseudo2Code to support future research on structured reasoning, interpretable code generation, and reliable AI-assisted software development.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-11 in cs.SE).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.09068 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:00:31.490Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).