Metraipolis › archive › dc_e4a6efb3142c
GNN-CB: A Graph Neural Network Competition Benchmark for Human and LLM Evaluation — arXiv:2610.05387
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.05387
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:26:54.190Z
- SHA-256 of the archived text
e7e94df452cc5db14bba7336eb58ea71bcc5745d44fe5fe74ce78e747ac20528- Stable URL of this page
https://metraipolis.com/documento/dc_e4a6efb3142c
How to cite
GNN-CB: A Graph Neural Network Competition Benchmark for Human and LLM Evaluation — arXiv:2610.05387 — arXiv, https://arxiv.org/abs/2610.05387 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e4a6efb3142c (Metraipolis archive dc_e4a6efb3142c, sha256 e7e94df452cc5db14bba7336eb58ea71bcc5745d44fe5fe74ce78e747ac20528)
BibTeX
@misc{dce4a6efb3142c,
title = {GNN-CB: A Graph Neural Network Competition Benchmark for Human and LLM Evaluation — arXiv:2610.05387},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.05387},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e4a6efb3142c; sha256 e7e94df452cc5db14bba7336eb58ea71bcc5745d44fe5fe74ce78e747ac20528},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05387. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) have demonstrated strong performance on coding and reasoning benchmarks; however, their ability to solve graph-structured machine learning problems remains largely unexplored. In particular, no benchmark currently evaluates whether LLMs can autonomously solve end-to-end Graph Neural Network (GNN) coding tasks under realistic competition settings. To address this gap, this paper introduces GNN-CB, the first competition-based benchmark for evaluating both humans and LLMs on GNN coding tasks. GNN-CB consists of 18 curated competitions spanning node-, edge-, and graph-level prediction across diverse graph categories, domains, and difficulty tiers. All submissions are evaluated through a unified automated pipeline with hidden test sets and standardized scoring. Human participants solve tasks under controlled competition constraints, while LLMs are evaluated using a frozen zero-shot prompting protocol based on a plan-then-code paradigm with bounded execute-and-repair loops. The benchmark additionally supports both non-agent and autonomous agent-based evaluation within the same protocol. Under our evaluated protocol, LLMs rarely match Human Top performance and show less stable performance across competitions. No single model dominates: a few competitions are won by LLMs, yet humans still hold the top score on most tasks. We release GNN-CB as a living benchmark with automated evaluation infrastructure, dynamic leaderboards, and reproducible execution pipelines. Beyond benchmarking, GNN-CB provides a practice-oriented resource for studying GNN implementation across progressively diverse graph-learning tasks. The benchmark and evaluation framework are publicly available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2610.05387 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:06.019Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).