Metraipolis › archive › dc_ff9de7b24bea

ConvexLLM: Scaling Convex-Hull Speech Interfaces to General-Purpose Audio Intelligence — arXiv:2610.23539

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.23539
License (as archived)
CC0-1.0
Archived
2026-10-19T08:43:53.335Z
SHA-256 of the archived text
b0c3f6fdd22b65c2fedd8096644d2c1f7464f335b6abbbaee9eae7a5183c7243
Stable URL of this page
https://metraipolis.com/documento/dc_ff9de7b24bea

How to cite

ConvexLLM: Scaling Convex-Hull Speech Interfaces to General-Purpose Audio Intelligence — arXiv:2610.23539 — arXiv (en), https://arxiv.org/abs/2610.23539 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ff9de7b24bea (Metraipolis archive dc_ff9de7b24bea, sha256 b0c3f6fdd22b65c2fedd8096644d2c1f7464f335b6abbbaee9eae7a5183c7243)
BibTeX
@misc{dcff9de7b24bea,
  title = {ConvexLLM: Scaling Convex-Hull Speech Interfaces to General-Purpose Audio Intelligence — arXiv:2610.23539},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.23539},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ff9de7b24bea; sha256 b0c3f6fdd22b65c2fedd8096644d2c1f7464f335b6abbbaee9eae7a5183c7243},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.23539. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speech and audio language models connect continuous acoustic representations to decoders pretrained primarily on text data. Most such models use MLP or Q-Former interfaces to map acoustic features into the decoder's input space, requiring the decoder to learn how to interpret the resulting representations. We investigate whether grounding these representations in the decoder's existing text embeddings can help it use acoustic information while better preserving its pretrained text capabilities. We present ConvexLLM, a general-purpose speech and audio language model whose C-Gate interface represents audio as continuous convex combinations of frozen text embeddings. ConvexLLM achieves an eleven-task VoiceBench aggregate of 75.36, the highest among the open-weight models we compare, and an MMAU Test-mini accuracy of 62.50%. Its largest margins over the baselines are on spoken knowledge and reasoning tasks, while the strongest baselines are more accurate on audio understanding and transcription. In a comparison with a 2B-parameter decoder, C-Gate achieves higher aggregate spoken-language performance than MLP and Q-Former interfaces, with text-task performance close to MLP and task-dependent tradeoffs in transcription and audio understanding. These results support convex combinations of text embeddings as a viable interface for general-purpose speech and audio language models.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Sound (cs.SD); Audio and Speech Processing (eess.AS).
Pagina del lavoro: https://arxiv.org/abs/2610.23539 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:24.875Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).