Metraipolis › archive › dc_bdcdb40a1a88
DLER: Dual-Logit Experience Replay for Online Continual Learning — arXiv:2611.04885
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04885
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T21:51:47.830Z
- SHA-256 of the archived text
6d67b61383d36557fdcc5266be4315c435cf7a848c8f0d7895cf31d9ee5c43d9- Stable URL of this page
https://metraipolis.com/documento/dc_bdcdb40a1a88
How to cite
DLER: Dual-Logit Experience Replay for Online Continual Learning — arXiv:2611.04885 — arXiv (en), https://arxiv.org/abs/2611.04885 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bdcdb40a1a88 (Metraipolis archive dc_bdcdb40a1a88, sha256 6d67b61383d36557fdcc5266be4315c435cf7a848c8f0d7895cf31d9ee5c43d9)
BibTeX
@misc{dcbdcdb40a1a88,
title = {DLER: Dual-Logit Experience Replay for Online Continual Learning — arXiv:2611.04885},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04885},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bdcdb40a1a88; sha256 6d67b61383d36557fdcc5266be4315c435cf7a848c8f0d7895cf31d9ee5c43d9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04885. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Online continual learning aims to continuously train neural networks on a non-stationary data stream in a single pass. Rehearsal-based methods alleviate catastrophic forgetting by replaying stored samples, but the imbalance between current and previous data often leads to predictions biased toward current classes. Many approaches have been proposed to overcome the forgetting problem by correcting the bias; however, they still need to be improved in an online fashion. By decomposing the dot-product logits into an angle factor and a norm factor, our analysis reveals that the angle factor efficiently captures novel knowledge but is particularly susceptible to prediction bias, whereas the norm factor helps preserve historical knowledge. Based on these observations, we propose Dual-Logit Experience Replay (DLER), which exploits dual-logits for both model optimization and memory management. Specifically, DLER learns current samples using cosine logits and incorporates additional dot-product logits when replaying previous samples to mitigate prediction bias. To strengthen dual-logit learning under limited replay memory, an auxiliary supervised contrastive objective promotes angular compactness within each class, extending corrections learned from limited replay samples to other samples of the same class. Meanwhile, we introduce a memory update strategy that preferentially retains samples for which dot-product prediction assigns a higher probability to the correct class than cosine prediction, aligning sample selection with dual-logit training. Extensive experiments on three datasets demonstrate the effectiveness of DLER, particularly under limited memory budgets. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.04885 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T21:38:49.736Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).