Metraipolis › archive › dc_ac545d900a2b

Representation-Aligned Auxiliary Supervision for Language Model Adaptation — arXiv:2610.04098

Source
arXiv
Original
https://arxiv.org/abs/2610.04098
License (as archived)
CC0-1.0
Archived
2026-10-06T05:28:01.092Z
SHA-256 of the archived text
47a8fe199a33426aaacdfa68d0dc4831f76bab5e8424d1e23526ff01bde85284
Stable URL of this page
https://metraipolis.com/documento/dc_ac545d900a2b

How to cite

Representation-Aligned Auxiliary Supervision for Language Model Adaptation — arXiv:2610.04098 — arXiv, https://arxiv.org/abs/2610.04098 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ac545d900a2b (Metraipolis archive dc_ac545d900a2b, sha256 47a8fe199a33426aaacdfa68d0dc4831f76bab5e8424d1e23526ff01bde85284)
BibTeX
@misc{dcac545d900a2b,
  title = {Representation-Aligned Auxiliary Supervision for Language Model Adaptation — arXiv:2610.04098},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.04098},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ac545d900a2b; sha256 47a8fe199a33426aaacdfa68d0dc4831f76bab5e8424d1e23526ff01bde85284},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04098. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Language models exhibit strong reasoning capabilities, yet adapting them to structured domains remains challenging and can yield inconsistent outcomes. We identify representation compatibility, the extent to which a model effectively processes a representation for a structured task, as a key factor in adaptation. We study this in chess, which provides a controlled testbed with precise semantics, computable optimal actions, and multiple state representations, including a symbolic encoding (FEN) and a spatial format (ASCII). We find that models often process semantically equivalent inputs substantially differently, affecting both learning and generalization. Building on this observation, we propose representation-aligned auxiliary supervision, which uses environment-derived tasks expressed in compatible representations to improve adaptation to structured domains. Across models and representations, auxiliary supervision consistently improves optimal-move prediction relative to target-only training under identical target data. Tasks that expose environment dynamics provide larger and most consistent gains than surface-level or static supervision, while remaining competitive with substantially increasing the amount of target-task data. Moreover, ASCII-trained models transfer more effectively to FEN than FEN-trained models do to ASCII, even surpassing the FEN target-only baseline on FEN evaluation. The gains also extend beyond optimal-move prediction to open-ended, factually grounded commentary generation. Overall, our results show that auxiliary supervision in model-compatible representations can enable effective adaptation in structured domains.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.04098 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:20.248Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).