Metraipolis › archive › dc_540f6dbe421c

Measuring Where Intent Gets Lost in Agentic Smart Contract Synthesis — arXiv:2610.37830

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.37830
License (as archived)
CC0-1.0
Archived
2026-10-31T06:06:44.428Z
SHA-256 of the archived text
2a1e309eb9aa9734525826ddb3ea5c90881c66306e7f51339d4e8572cd1cdadb
Stable URL of this page
https://metraipolis.com/documento/dc_540f6dbe421c

How to cite

Measuring Where Intent Gets Lost in Agentic Smart Contract Synthesis — arXiv:2610.37830 — arXiv (en), https://arxiv.org/abs/2610.37830 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_540f6dbe421c (Metraipolis archive dc_540f6dbe421c, sha256 2a1e309eb9aa9734525826ddb3ea5c90881c66306e7f51339d4e8572cd1cdadb)
BibTeX
@misc{dc540f6dbe421c,
  title = {Measuring Where Intent Gets Lost in Agentic Smart Contract Synthesis — arXiv:2610.37830},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.37830},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_540f6dbe421c; sha256 2a1e309eb9aa9734525826ddb3ea5c90881c66306e7f51339d4e8572cd1cdadb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.37830. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI agents can now write software from a plain-English description, but code that runs is not the same as code that does what was asked. This gap matters most for smart contracts, the programs behind Web3 applications that move money on a blockchain and cannot be fixed once deployed. We ask how to check whether an AI-written contract behaves as it should, and where the requested behavior gets lost. We design an evaluation built on execution rather than opinion. We replay the same transactions on an AI-written contract and on the original it should reproduce and measure how often they agree. We test every grading tool on planted mistakes, compare against an answer key written without the description, and audit our own scorer until known wrong answers cannot pass. Common methods, such as AI judges and standard analysis tools, miss obvious mistakes, and scoring choices matter too. Checking only the functions every system includes hid about half the difference between pipelines. We then propose an elicitation-first architecture. Before writing code, the agent asks the client a short checklist of questions, checks the contract against an independently written executable answer key, and returns disagreements to the client instead of silently rewriting code. Asking first matters far more than reviewing or patching afterwards, across three AI models, while automated security fixes often break requested behavior. A few well-chosen questions recover much of what a short description omits, and their value can be estimated without the original contract. To our knowledge, this is the first study in smart contract synthesis to gather missing requirements before generation and check the result against an independently written executable model. The lessons extend to any setting where AI agents turn requests into programs. We release our tools, 600 tasks and all results.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MA, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Human-Computer Interaction (cs.HC); Information Retrieval (cs.IR); Multiagent Systems (cs.MA); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2610.37830 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:06:38.419Z dalla pagina https://arxiv.org/list/cs.MA/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).