Metraipolis › archive › dc_e2a62aff2e6c

Beyond Leaderboard Scores: A Deployment-Focused Protocol for Interpretable Tracking Evaluation in Pedestrian-Centric Environments — arXiv:2610.01682

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.01682
License (as archived)
CC0-1.0
Archived
2026-10-06T19:16:50.066Z
SHA-256 of the archived text
dc19c8f20ebc01ee0bba7c0f5c394aad1c2110d1b36a9b6f3a17878a63e41c56
Stable URL of this page
https://metraipolis.com/documento/dc_e2a62aff2e6c

How to cite

Beyond Leaderboard Scores: A Deployment-Focused Protocol for Interpretable Tracking Evaluation in Pedestrian-Centric Environments — arXiv:2610.01682 — arXiv (en), https://arxiv.org/abs/2610.01682 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e2a62aff2e6c (Metraipolis archive dc_e2a62aff2e6c, sha256 dc19c8f20ebc01ee0bba7c0f5c394aad1c2110d1b36a9b6f3a17878a63e41c56)
BibTeX
@misc{dce2a62aff2e6c,
  title = {Beyond Leaderboard Scores: A Deployment-Focused Protocol for Interpretable Tracking Evaluation in Pedestrian-Centric Environments — arXiv:2610.01682},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.01682},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e2a62aff2e6c; sha256 dc19c8f20ebc01ee0bba7c0f5c394aad1c2110d1b36a9b6f3a17878a63e41c56},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01682. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Mobile robots operating among pedestrians need trajectories that become available quickly, remain spatially credible through missed observations, preserve identity, and fit within an embedded computing budget. Aggregate tracking scores provide limited insight into when and how trajectories fail, while varying detector inputs can confound tracker and detector quality. We present a deployment-focused, tracker-only evaluation protocol that uses shared detections to isolate tracker behavior and directly evaluates initialization, detector-gap continuation, identity recovery, close-neighbor association, and load-dependent tracker-step runtime, while Higher Order Tracking Accuracy (HOTA) is retained as a complementary aggregate measure. We apply the protocol to the JackRabbot Dataset and Benchmark (JRDB) using six open-source trackers and our lightweight Pedestrian Reference Tracker (PedRefTrack), together with a GT-assisted variant that estimates the remaining tracker-side gap under idealized association and motion. Under fixed detections, the non-GT trackers span only 24.26%-29.67% HOTA yet exhibit markedly different capability profiles. After 1.0 s without detector support, no tracker without GT assistance maintains spatially correct, same-identity output in more than half of eligible cases, making missing-observation continuation the dominant limitation among the tested properties. Close-neighbor failures are smaller and increase mainly at the shortest separations. Tracker-step runtime on an NVIDIA Jetson Orin is heavy-tailed and load-sensitive, causing several trackers to fall below the 10 Hz real-time target in crowded frames. The protocol provides a reproducible way to characterize tracker behavior and deployment suitability in pedestrian-centric environments. Code and evaluation scripts are released at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-02 in cs.CV).
Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.01682 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T17:37:36.951Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).