Metraipolis › archive › dc_44ac7964d112

Reuse or Relearn? A Spectral View of Earth Observation Foundation Models — arXiv:2609.32756

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32756
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:49.369Z
SHA-256 of the archived text
767b7c91593a8632659e8448967565d3bc2671019726226ddd54505001144f52
Stable URL of this page
https://metraipolis.com/documento/dc_44ac7964d112

How to cite

Reuse or Relearn? A Spectral View of Earth Observation Foundation Models — arXiv:2609.32756 — arXiv (en), https://arxiv.org/abs/2609.32756 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_44ac7964d112 (Metraipolis archive dc_44ac7964d112, sha256 767b7c91593a8632659e8448967565d3bc2671019726226ddd54505001144f52)
BibTeX
@misc{dc44ac7964d112,
  title = {Reuse or Relearn? A Spectral View of Earth Observation Foundation Models — arXiv:2609.32756},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32756},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_44ac7964d112; sha256 767b7c91593a8632659e8448967565d3bc2671019726226ddd54505001144f52},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32756. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Foundation models are rarely used as generic, frozen feature extractors; instead, they are fine-tuned for the target downstream application. This practice is particularly prevalent in Earth observation (EO), and it raises a question that downstream accuracy alone cannot answer: does fine-tuning reuse the pretrained representation, or does it relearn a new one? We study this with spectral diagnostics that compare a model before and after adaptation, quantifying how well its dominant singular subspaces are preserved, how broadly the weight update is distributed, and how large it is. Using natural image models such as CLIP and DINO as a reference, we find that, under the evaluated fine-tuning settings, EO models undergo far larger, higher-rank updates and retain much less of their pretrained structure, so their downstream performance is often obtained with substantial changes to the pretrained weight structure. The diagnostics further provide insight into how cheaply a model can be adapted: where the pretrained subspaces are preserved, adapting a small fraction of the parameters can match full fine-tuning, and where they are not, it can fall behind. More broadly, foundation models, and EO foundation models in particular, should be assessed not only by benchmark accuracy, but also by how reusable their pretrained representation is.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32756 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.756Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).