Metraipolis › archive › dc_074b094e13e8

EnterpriseSynth: Zero-Execution SFT and Evaluation Data from OpenAPI Specs — arXiv:2610.22412

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.22412
License (as archived)
CC0-1.0
Archived
2026-10-19T09:37:10.449Z
SHA-256 of the archived text
17037cb2f2faebac7445d9f385b4356b3699befe45251bfdb5dabeb5b6b86136
Stable URL of this page
https://metraipolis.com/documento/dc_074b094e13e8

How to cite

EnterpriseSynth: Zero-Execution SFT and Evaluation Data from OpenAPI Specs — arXiv:2610.22412 — arXiv (en), https://arxiv.org/abs/2610.22412 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_074b094e13e8 (Metraipolis archive dc_074b094e13e8, sha256 17037cb2f2faebac7445d9f385b4356b3699befe45251bfdb5dabeb5b6b86136)
BibTeX
@misc{dc074b094e13e8,
  title = {EnterpriseSynth: Zero-Execution SFT and Evaluation Data from OpenAPI Specs — arXiv:2610.22412},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22412},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_074b094e13e8; sha256 17037cb2f2faebac7445d9f385b4356b3699befe45251bfdb5dabeb5b6b86136},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22412. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Tool-using large language model agents require substantial training and evaluation data, yet collecting execution traces from enterprise APIs is often unsafe, expensive, or infeasible because of access controls, compliance constraints, rate limits, and the risk of modifying production data. We present EnterpriseSynth, a zero-execution framework that converts OpenAPI specifications into supervised fine-tuning trajectories and paired evaluation instances without invoking the underlying APIs. The framework uses a four-stage pipeline for specification parsing, intent generation, trajectory synthesis, and static validation. Each generated trajectory is checked against the source specification for endpoint, parameter, type, and schema consistency before being admitted to the dataset. We evaluate EnterpriseSynth on 17 real and synthetic APIs. Adversarial refinement increases the validator's detection rate for planted schema violations from 57-80% to 98% (43/44). Across five random seeds, a compact open-weight model fine-tuned on EnterpriseSynth data outperforms both an untuned model and a Self-Instruct baseline on held-out APIs. Performance remains consistent on five private, hand-authored API specifications that were not available during training, with tool-selection accuracy of 40.0% on the private set and 39.6% on the public held-out set. However, an independent LLM-based evaluation shows that endpoint-selection accuracy substantially overestimates end-to-end correctness: accuracy falls from 48.9% under the endpoint-level metric to 21.3% when argument selection and values are also evaluated. These results demonstrate that OpenAPI specifications can support scalable, execution-free data generation, while highlighting the need for argument-level evaluation when assessing the deployment readiness of tool-using agents.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.SE, nuovi lavori del giorno).
Categorie arXiv: Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2610.22412 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:31:59.693Z dalla pagina https://arxiv.org/list/cs.SE/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).