Metraipolis › archive › dc_1f24d033ee70

Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding — arXiv:2608.04127

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04127
License (as archived)
CC0-1.0
Archived
2026-10-31T11:47:45.892Z
SHA-256 of the archived text
18d6dd7ffb3ea09009ae642b35c09bbecac3df73687286bd95e957ad4970b88b
Stable URL of this page
https://metraipolis.com/documento/dc_1f24d033ee70

How to cite

Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding — arXiv:2608.04127 — arXiv (en), https://arxiv.org/abs/2608.04127 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1f24d033ee70 (Metraipolis archive dc_1f24d033ee70, sha256 18d6dd7ffb3ea09009ae642b35c09bbecac3df73687286bd95e957ad4970b88b)
BibTeX
@misc{dc1f24d033ee70,
  title = {Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding — arXiv:2608.04127},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04127},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1f24d033ee70; sha256 18d6dd7ffb3ea09009ae642b35c09bbecac3df73687286bd95e957ad4970b88b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04127. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language model agents need to perceive human behavior in physical environments. Millimeter-wave (mmWave) radar provides a privacy-friendly and contactless sensing modality, but radar observations are difficult to align with language. Existing radar-language methods often rely on synthetic data or lack explicit supervision for human body structure and motion. We present mmMind, a radar-language model that uses synchronized 3D pose as training-only supervision. A spatio-temporal radar encoder is pretrained to capture body configuration and motion dynamics, after which the pose head is removed so that inference requires radar alone. The learned radar representations are then aligned with an LLM for behavior captioning and spatio-temporal question answering. We also introduce mmMind-Bench, a real-world mmWave-language benchmark containing 17.9 hours of recordings from 23 participants across seven indoor environments. Experiments on captioning, question answering, and unseen-action generalization show that mmMind consistently outperforms existing radar-language baselines, while ablations confirm the importance of pose-guided pretraining.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-06 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.04127 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:37:40.645Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).