Metraipolis › archive › dc_ffbd4d41dbf1

Event-Driven ML Pipeline Orchestration for Manufacturing: An AWS Industry Experience — arXiv:2610.06890

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.06890
License (as archived): CC0-1.0
Archived: 2026-10-07T11:40:38.196Z
SHA-256 of the archived text: c26ef8686f54a82834855e708a6d81b904dcdbd115c68b0bc7261a218d07d7bc
Stable URL of this page: https://metraipolis.com/documento/dc_ffbd4d41dbf1

How to cite

Event-Driven ML Pipeline Orchestration for Manufacturing: An AWS Industry Experience — arXiv:2610.06890 — arXiv (en), https://arxiv.org/abs/2610.06890 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ffbd4d41dbf1 (Metraipolis archive dc_ffbd4d41dbf1, sha256 c26ef8686f54a82834855e708a6d81b904dcdbd115c68b0bc7261a218d07d7bc)

BibTeX

@misc{dcffbd4d41dbf1,
  title = {Event-Driven ML Pipeline Orchestration for Manufacturing: An AWS Industry Experience — arXiv:2610.06890},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.06890},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ffbd4d41dbf1; sha256 c26ef8686f54a82834855e708a6d81b904dcdbd115c68b0bc7261a218d07d7bc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.06890. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We present an industry experience report on three years of operating an event-driven cloud infrastructure for continuous machine learning training in automotive manufacturing. Our system orchestrates GPU-accelerated training of product-specialized model pairs, a physics prediction model and a reinforcement-learning control policy, across multiple plants, coordinating long-running GPU workloads triggered by manufacturing events. The architecture combines Amazon ECS with EC2 GPU capacity providers, SQS-based messaging with dead-letter queues, and an admission-controlled Lambda dispatcher that enforces cluster concurrency limits. A Conductor orchestrator on ECS Fargate initiates dependency-aware retraining chains on a weekly schedule. The entire infrastructure is codified in modular Terraform with multi-account separation. From 40000+ production training jobs we report a 72-78% cost reduction versus always-on GPU infrastructure. A discrete-event simulation confirms that admission control is necessary (naive dispatch loses 65% of jobs) and that queue-draining matches AWS Step Functions latency while eliminating per-job startup overhead. We provide lessons learned and release the simulator and Terraform module skeletons as open-source artifacts.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.06890 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:19:24.704Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).