Metraipolis › archive › dc_a9bd92e175df

Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads — arXiv:2608.11661

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.11661
License (as archived)
CC0-1.0
Archived
2026-10-19T05:16:03.113Z
SHA-256 of the archived text
37680acf3ed38c8f52642e9f736ee7d7800a4c7c0b98bfc43a8f56ea1c3611bf
Stable URL of this page
https://metraipolis.com/documento/dc_a9bd92e175df

How to cite

Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads — arXiv:2608.11661 — arXiv (en), https://arxiv.org/abs/2608.11661 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a9bd92e175df (Metraipolis archive dc_a9bd92e175df, sha256 37680acf3ed38c8f52642e9f736ee7d7800a4c7c0b98bfc43a8f56ea1c3611bf)
BibTeX
@misc{dca9bd92e175df,
  title = {Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads — arXiv:2608.11661},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11661},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a9bd92e175df; sha256 37680acf3ed38c8f52642e9f736ee7d7800a4c7c0b98bfc43a8f56ea1c3611bf},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11661. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings. This architecture has been developed independently in operator learning, bipartite matching, contrastive vision-language models, retrieval, and other areas, yet no unified theory guides the basic design decisions: how many interaction modes to represent, how to normalize the encoders, and when the architecture should be avoided. We provide such a foundation by introducing the class of functions of low interaction rank, a class whose intrinsic complexity is measured by its interaction spectrum. Within this framework, approximation error decomposes into a spectral truncation term and an encoder-realization term; sample complexity is governed by the sum of the two encoder complexities rather than their product; and a usability criterion based on spectral decay determines when the architecture can succeed. The same framework exposes a central identifiability problem: the encoders are defined only up to a linear gauge symmetry that leaves the learned coordinates arbitrary. We show that normalization is gauge fixing and that whitening pins the interaction modes up to permutation and sign, thereby explaining the uninterpretability of contrastive dimensions and providing a constructive remedy. Experiments on synthetic kernels, operator learning, and CLIP models validate the theoretical predictions: spectral decay rates match the predicted scaling, whitening recovers the true modes, and independently trained CLIP models are related by a single rotation which, after removal by whitening, exposes interpretable concept axes. The code of this paper is provided at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.11661 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:08:46.712Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).