Metraipolis › archive › dc_8d7dcaa216e9
Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating — arXiv:2609.03900
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.03900
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-17T02:08:21.843Z
- SHA-256 of the archived text
76ac18fafd449c6ab61cee0a9b48c903b3b670c1fd1ead1b1d02db8b7642f7ab- Stable URL of this page
https://metraipolis.com/documento/dc_8d7dcaa216e9
How to cite
Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating — arXiv:2609.03900 — arXiv (en), https://arxiv.org/abs/2609.03900 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8d7dcaa216e9 (Metraipolis archive dc_8d7dcaa216e9, sha256 76ac18fafd449c6ab61cee0a9b48c903b3b670c1fd1ead1b1d02db8b7642f7ab)
BibTeX
@misc{dc8d7dcaa216e9,
title = {Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating — arXiv:2609.03900},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.03900},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8d7dcaa216e9; sha256 76ac18fafd449c6ab61cee0a9b48c903b3b670c1fd1ead1b1d02db8b7642f7ab},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.03900. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Continual knowledge-updating methods are often declared superior from one final checkpoint and one conventional adapter rank. We show that this can be insufficient to identify the better operating point. Holding a periodic hierarchy fixed, we compare it with cumulative replay over a 24-month Wikidata stream while varying evaluation month, replay LoRA rank, and query formulation. The apparent winner changes across this region: on Qwen2.5-1.5B, the hierarchy's 5.0-point advantage over rank-8 replay becomes an 11.6-point deficit against rank-72 replay, and at high ranks a consolidation-aligned endpoint can suggest a tie while time-averaged replay leads by 9-13 points. The same rank-conditioned reversal appears on Llama-3.2-1B and held-out paraphrases. These results show that method ranking in continual updating can depend jointly on when performance is measured and how much replay-side adaptation capacity the baseline receives. We therefore propose reporting trajectories and capacity sweeps, and declaring a robust winner only when the ordering is stable across the evaluation region; otherwise, comparisons should report winner regions and retention-stability-cost frontiers. Under this protocol, the periodic hierarchy is a lower-update-cost operating point, not a quality winner. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-04 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.03900 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-17T01:47:25.577Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-04?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).