Metraipolis › archive › dc_b34a423dfab2

Can LLM Agents Be Forward-Deployed Engineers? Requirement Discovery and Delivery for Enterprise Text-to-SQL — arXiv:2611.03551

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03551
License (as archived)
CC0-1.0
Archived
2026-11-03T10:38:17.264Z
SHA-256 of the archived text
d995577d947731066d9411b0b949cc868f75ae5c5fb450e968b66fd033be5df3
Stable URL of this page
https://metraipolis.com/documento/dc_b34a423dfab2

How to cite

Can LLM Agents Be Forward-Deployed Engineers? Requirement Discovery and Delivery for Enterprise Text-to-SQL — arXiv:2611.03551 — arXiv (en), https://arxiv.org/abs/2611.03551 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b34a423dfab2 (Metraipolis archive dc_b34a423dfab2, sha256 d995577d947731066d9411b0b949cc868f75ae5c5fb450e968b66fd033be5df3)
BibTeX
@misc{dcb34a423dfab2,
  title = {Can LLM Agents Be Forward-Deployed Engineers? Requirement Discovery and Delivery for Enterprise Text-to-SQL — arXiv:2611.03551},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03551},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b34a423dfab2; sha256 d995577d947731066d9411b0b949cc868f75ae5c5fb450e968b66fd033be5df3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03551. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM coding agents can increasingly automate model post-training, but whether they can act as forward-deployed engineer (FDE) agents, discovering customer requirements and turning them into deployment decisions, remains unclear. We study this on a real enterprise text-to-SQL engagement: a simulated customer, grounded in the real use case's requirements, starts with an ambiguous request. An agent must discover acceptance criteria, data permissions, and hardware constraints in the discovery phase, then plan and execute a post-training recipe in the delivery phase. Across 15 models, the strongest discover over 80% of withheld requirements and open-weight models discover 40-50%; in the delivery phase, only 25% of runs across eight coding-agent configurations pass all acceptance criteria. We identify four bottlenecks in this engagement: (1) incomplete discovery, with 40% of withheld requirements missing or misrepresented; (2) unreliable execution; (3) underused compute budgets and a lack of systematic comparisons across base models and post-training methods; and (4) benchmark accuracy does not guarantee production readiness. We suggest human review of discovered requirements, monitoring of the agent's plan during execution, and an explicit experiment-environment specification covering hardware, budget scope, and production checks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.03551 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.953Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).