Metraipolis › archive › dc_7b6ea1b56353

MMPostTrainBench: Benchmarking Autonomous Research for Multimodal Post-Training — arXiv:2610.05398

Source
arXiv
Original
https://arxiv.org/abs/2610.05398
License (as archived)
CC0-1.0
Archived
2026-10-06T05:25:09.331Z
SHA-256 of the archived text
636f93576beb52c9dbdf7a8b028620bdaeaecb41c345899ae4287122f1e373d6
Stable URL of this page
https://metraipolis.com/documento/dc_7b6ea1b56353

How to cite

MMPostTrainBench: Benchmarking Autonomous Research for Multimodal Post-Training — arXiv:2610.05398 — arXiv, https://arxiv.org/abs/2610.05398 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7b6ea1b56353 (Metraipolis archive dc_7b6ea1b56353, sha256 636f93576beb52c9dbdf7a8b028620bdaeaecb41c345899ae4287122f1e373d6)
BibTeX
@misc{dc7b6ea1b56353,
  title = {MMPostTrainBench: Benchmarking Autonomous Research for Multimodal Post-Training — arXiv:2610.05398},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05398},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7b6ea1b56353; sha256 636f93576beb52c9dbdf7a8b028620bdaeaecb41c345899ae4287122f1e373d6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05398. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Autonomous research seeks sustained model improvements through iterative experimentation and feedback. LLM agents show promise in automating machine learning and language-model post-training, but their ability to sustain multimodal improvement remains unclear. We introduce MMPostTrainBench, a benchmark spanning eight tasks in image, audio, video, and joint audio-video understanding and image-grounded software repair. Agents operate from a common base model within fixed budgets, using development feedback before independent evaluation of their submitted models. Evaluation covers target and non-target model outcomes, iterative model improvement and selection, and research integrity. Across all eight tasks, 52.1% of model--task means fall below the base, and evaluated submissions also exhibit non-target regressions. Model performance does not consistently improve across research iterations, and agents do not reliably select the best evaluated candidate for submission; final submissions trail that candidate by up to 5.38 percentage points. Extending autonomous research from text-only to multimodal tasks introduces additional sources of error in perception, cross-modal alignment, and temporal grounding. The observed regressions and selection gaps highlight the need to balance targeted improvements with non-target capability preservation and to retain gains across research iterations. These requirements motivate MMResearch, a multimodal research framework that connects media-grounded evidence to hypotheses and interventions, carries findings across rounds through hierarchical memory, and retains candidates using development evaluation. Added to existing code-agent runtimes, it improves submitted-model accuracy by up to 7.75 percentage points for Claude Opus 4.8 with Claude Code and 2.33 points for GPT-5.6-sol with Codex.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.05398 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.583Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).