Metraipolis › archive › dc_75fda03498c0

Drive by Hindsight and Foresight: Tool-Grounded Synergistic Reasoning over Hierarchical Memory for Autonomous Driving — arXiv:2609.08217

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.08217
License (as archived)
CC0-1.0
Archived
2026-10-16T15:12:07.645Z
SHA-256 of the archived text
05fcd363166e4d12a84371e542cb4ec430691a0e7e9af5730fb12edcca4b2558
Stable URL of this page
https://metraipolis.com/documento/dc_75fda03498c0

How to cite

Drive by Hindsight and Foresight: Tool-Grounded Synergistic Reasoning over Hierarchical Memory for Autonomous Driving — arXiv:2609.08217 — arXiv (en), https://arxiv.org/abs/2609.08217 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_75fda03498c0 (Metraipolis archive dc_75fda03498c0, sha256 05fcd363166e4d12a84371e542cb4ec430691a0e7e9af5730fb12edcca4b2558)
BibTeX
@misc{dc75fda03498c0,
  title = {Drive by Hindsight and Foresight: Tool-Grounded Synergistic Reasoning over Hierarchical Memory for Autonomous Driving — arXiv:2609.08217},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.08217},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_75fda03498c0; sha256 05fcd363166e4d12a84371e542cb4ec430691a0e7e9af5730fb12edcca4b2558},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.08217. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

VLMs have shown promise for autonomous driving, yet still suffer from hallucination, weak spatio-temporal perception, and limited generalization. Recent methods improve reasoning and decision-making through CoT explanations, retrieval-augmented generation or the static injection of tool outputs. Although these mechanisms enrich the context, the model neither proactively perceives scene information nor accumulates experience after answering. To overcome these limitations, we present, to our knowledge, the first synergistic framework that tightly couples hierarchical memory with proactive tool invocation in a closed reasoning loop. Our contributions are threefold. (i) Hierarchical Driving Memory: a scene-level short-term memory maintains the dynamic scene state, and an evolving long-term memory retrieves reusable experience and tool strategies. (ii) Memory-Tool Synergistic Reasoning Framework: guided by the scene state and retrieved experience, the model adaptively invokes tools to refine its reasoning at inference time and consolidates reusable experience into a long-term memory pool offline. (iii) Data Generation and Two-stage Training Pipeline: verified memory-tool trajectories built by multi-step teacher rollout are used to train with SFT and GRPO. Our 7B model reaches an overall reasoning score of 80.03 and MCQ accuracy of 79.09% on DriveLMM-o1, surpassing the strongest baseline by 7.74 MCQ points and generalizes strongly across benchmarks. Notably, ablation and analysis studies validate the effectiveness of each component and further reveal the complementary roles of hierarchical memory. Short-term memory strengthens spatio-temporal understanding, improving STSBench accuracy by 24.2 points, while offline long-term memory consolidation yields an additional 3.57-point MCQ gain with all parameters frozen, demonstrating continual self-evolution through accumulated driving experience.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.08217 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T14:49:57.288Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).