Metraipolis › archive › dc_bd2866fc3a5e

Towards Fine-Grained Object Manipulation: SAM3-Guided Visuomotor Policy with Persistent Memory Learning and Focused Visual Conditioning — arXiv:2609.21621

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.21621
License (as archived)
CC0-1.0
Archived
2026-10-10T02:30:14.827Z
SHA-256 of the archived text
0654c69626b3bfb9a17b29c484dce9657cb470c985817c3d5a0186881d79847f
Stable URL of this page
https://metraipolis.com/documento/dc_bd2866fc3a5e

How to cite

Towards Fine-Grained Object Manipulation: SAM3-Guided Visuomotor Policy with Persistent Memory Learning and Focused Visual Conditioning — arXiv:2609.21621 — arXiv (en), https://arxiv.org/abs/2609.21621 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bd2866fc3a5e (Metraipolis archive dc_bd2866fc3a5e, sha256 0654c69626b3bfb9a17b29c484dce9657cb470c985817c3d5a0186881d79847f)
BibTeX
@misc{dcbd2866fc3a5e,
  title = {Towards Fine-Grained Object Manipulation: SAM3-Guided Visuomotor Policy with Persistent Memory Learning and Focused Visual Conditioning — arXiv:2609.21621},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21621},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bd2866fc3a5e; sha256 0654c69626b3bfb9a17b29c484dce9657cb470c985817c3d5a0186881d79847f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21621. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Fine-grained object (FO) manipulation requires robots to distinguish a specified FO from visually similar objects and execute actions reliably despite scene distractors. However, scene-level visual conditioning lacks explicit object selection, while category-level guidance cannot reliably distinguish FOs within the same category. We present a SAM3-guided visuomotor policy framework that addresses these challenges through persistent object memory and focused visual conditioning. First, we introduce FO Memory-driven SAM3 (FOM-SAM3), which learns reusable FO memory tokens from limited multi-view registration images, while keeping SAM3 fully frozen. Through one-vs-rest learning, these tokens encode persistent memories for localizing target FOs and rejecting similar alternatives, which can be stored in a memory bank for reuse. Second, we propose Focused Spatial-Appearance Encoding (FSAE), which combines in-FO appearance features with explicit bounding boxes to condition action policies, including Diffusion Policy (DP) and Action Chunking with Transformers (ACT). The effectiveness of the proposed FOM-SAM3 is validated on the FO-30 dataset comprising 30 physical objects across four coarse categories. Across three real-robot FO manipulation tasks, our FOM-SAM3-guided policies demonstrated robustness against distractors, discriminative ability of similar FOs, and extendibility to new FOs.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-21 in cs.RO).
Categorie arXiv: Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2609.21621 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T02:09:55.362Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).