Metraipolis › archive › dc_8b98db25e376

How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL — arXiv:2609.11132

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.11132
License (as archived)
CC0-1.0
Archived
2026-10-16T05:54:54.336Z
SHA-256 of the archived text
f6592c0810cb3860561c9a8cd85c99390a9ca92b0618f4f88996eaf39254a413
Stable URL of this page
https://metraipolis.com/documento/dc_8b98db25e376

How to cite

How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL — arXiv:2609.11132 — arXiv (en), https://arxiv.org/abs/2609.11132 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8b98db25e376 (Metraipolis archive dc_8b98db25e376, sha256 f6592c0810cb3860561c9a8cd85c99390a9ca92b0618f4f88996eaf39254a413)
BibTeX
@misc{dc8b98db25e376,
  title = {How Wrong Can a Good Predictor Be? Diverging Updates with Vanishing Predictive KL — arXiv:2609.11132},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.11132},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8b98db25e376; sha256 f6592c0810cb3860561c9a8cd85c99390a9ca92b0618f4f88996eaf39254a413},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11132. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Accurate posterior prediction need not require accurate approximation of Bayesian updates. We prove that an unbounded gap between the update maps can coexist with vanishing predictive KL for every fixed finite $K\ge2$ in a stationary symmetric Gaussian HMM. Exact Bayesian mixing and an explicit deterministic radial filter act on the same $K-1$ belief coordinates. As $q\to0^+$, their separation in centered logits in the worst case grows at least linearly in the natural confidence scale $L_K(q)$, while their categorical $D_{\mathrm{KL}}(\mathrm{exact}\|\mathrm{radial})$ vanishes at the same explicit witness. Along stationary HMM trajectories, the expected terminal KL between filtered posteriors also converges to zero at $H(q)=\lceil-\log(q)/c\rceil+1$. Typical blocks without switches drive both filters into a common confidence cone, where softmax curvature suppresses their disagreement; a single Gaussian maximal event controls adaptive noise. A sweep with equally spaced Gaussians over $K\in\{2,4,8\}$ illustrates the opposing trends, and binary controls at long horizons compare saturating and nonsaturating recurrences. The result isolates two missing links between internal update gaps and predictive cost: the contribution of separating states to expected loss and decoder sensitivity. Thus even an unbounded internal update gap does not by itself certify predictive failure. The construction is fixed in $K$ and does not provide a universal criterion for when compression is harmless or characterize when internal gaps must incur task loss.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.11132 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:44:24.974Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).