Metraipolis › archive › dc_79dffe2efcc9

When Can Text Embeddings Replace Item Calibration? A Geometric Diagnostic for Semantic Loadings in Multidimensional Adaptive Testing — arXiv:2608.09058

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.09058
License (as archived): CC0-1.0
Archived: 2026-10-19T08:55:09.668Z
SHA-256 of the archived text: 0f1d771c89f9525d0493cf11c8f2180aeb5edbff09653b6d1509f850c311a383
Stable URL of this page: https://metraipolis.com/documento/dc_79dffe2efcc9

How to cite

When Can Text Embeddings Replace Item Calibration? A Geometric Diagnostic for Semantic Loadings in Multidimensional Adaptive Testing — arXiv:2608.09058 — arXiv (en), https://arxiv.org/abs/2608.09058 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_79dffe2efcc9 (Metraipolis archive dc_79dffe2efcc9, sha256 0f1d771c89f9525d0493cf11c8f2180aeb5edbff09653b6d1509f850c311a383)

BibTeX

@misc{dc79dffe2efcc9,
  title = {When Can Text Embeddings Replace Item Calibration? A Geometric Diagnostic for Semantic Loadings in Multidimensional Adaptive Testing — arXiv:2608.09058},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09058},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_79dffe2efcc9; sha256 0f1d771c89f9525d0493cf11c8f2180aeb5edbff09653b6d1509f850c311a383},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09058. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multidimensional item response theory relies on calibrated item parameters, such as discrimination and category threshold values, which are usually estimated from large samples of human test responses. This study investigates whether the directional loadings of these parameters can be recovered directly from item text using pre-trained sentence embeddings, avoiding the need for initial item calibration. Using the open-source IPIP Big-Five dataset ($n=19{,}719$; 50 items), we built a multidimensional computerized adaptive testing (CAT) simulation using D-optimal item selection. We compared three item loading sources: fitted graded response model parameters, semantic text embeddings, and a lexical baseline. In simulation, semantic embeddings recovered latent trait profiles almost as accurately as fitted parameters (correlation $0.825$ vs. $0.857$), performing noticeably better than simple word overlap ($0.752$). However, the embedding-based model produced inflated posterior variance, showing nearly four times higher measurement uncertainty despite accurate point estimates. We attribute this to collinearity across dimensions, as embedding-derived loadings pointed in similar directions across traits (condition number $137$ vs. $1.0$; mean trait cosine $0.90$). This outcome reflects the shared vocabulary common in personality items. We propose a simple diagnostic metric based on the loading matrix condition number to evaluate whether an item bank is suitable for text-derived loadings prior to testing.

Tema degli esploratori: governance digitale e beni comuni (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CY).
Categorie arXiv: Applications (stat.AP) ; Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2608.09058 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:51:53.771Z dalla pagina https://arxiv.org/catchup/cs.CY/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).