Metraipolis › archive › dc_e450beda2dbb

WildHSR: Metric Feed-Forward 4D People-Scene Reconstruction from a 3D Foundation Model — arXiv:2609.29106

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.29106
License (as archived): CC0-1.0
Archived: 2026-10-08T04:27:17.867Z
SHA-256 of the archived text: 2e9b055d2d01c4e6ee6cb7e526c0d90b67ebd841ff45ce29efacc9f3b6e78389
Stable URL of this page: https://metraipolis.com/documento/dc_e450beda2dbb

How to cite

WildHSR: Metric Feed-Forward 4D People-Scene Reconstruction from a 3D Foundation Model — arXiv:2609.29106 — arXiv (en), https://arxiv.org/abs/2609.29106 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e450beda2dbb (Metraipolis archive dc_e450beda2dbb, sha256 2e9b055d2d01c4e6ee6cb7e526c0d90b67ebd841ff45ce29efacc9f3b6e78389)

BibTeX

@misc{dce450beda2dbb,
  title = {WildHSR: Metric Feed-Forward 4D People-Scene Reconstruction from a 3D Foundation Model — arXiv:2609.29106},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29106},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e450beda2dbb; sha256 2e9b055d2d01c4e6ee6cb7e526c0d90b67ebd841ff45ce29efacc9f3b6e78389},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29106. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

3D foundation models recover video cameras and geometry in one forward pass, but some of the strongest are up to scale. Joint people-scene reconstruction then requires two missing outputs: metric scale and persistent person identity. We ask whether one up-to-scale foundation representation can support both through lightweight adaptation. Exact metric labels are scarce, but unlabeled in-the-wild video is abundant. We use people in curated web video to initialise the solution: a posed metric body and 2D keypoints give an approximate, closed-form scale pseudo-label. These pseudo-labels pretrain a Scale Readout, which is then fine-tuned together with a lightweight adapter using exact metric supervision from standard real-video training splits. At inference the head predicts metric scale from foundation-model tokens, without the ruler or its teachers. For person identity, we probe the pretrained foundation model alone and find evidence that its intermediate query-key features encode person correspondence across frames. In most evaluated moving-person clips, a mid-layer token prefers that person over the vacated location and other people. A tiny projection reads this correspondence; together with metric pelvis motion and proposal confidence, it drives dustbin-aware Sinkhorn association of per-frame bodies. WildHSR combines both readouts to reconstruct metric cameras, scene and people from monocular video. Each window is predicted feed-forward; analytic association and Sim(3) composition connect windows. On EMDB-2, WildHSR is the first feed-forward method in the published comparison to beat the best optimization-based WA-MPJPE and RTE while leading feed-forward methods on all three world-frame metrics. On RICH, it leads feed-forward people-and-scene methods on WA-MPJPE and W-MPJPE. The complete pipeline runs at 10.1 fps on one GPU.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Computational Geometry (cs.CG).
Pagina del lavoro: https://arxiv.org/abs/2609.29106 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:58.007Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).