Metraipolis › archive › dc_2963541f4346
Where A Small Language Model Helps in Invoice Categorisation, Understood Through Embedding Geometry — arXiv:2608.18033
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.18033
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T14:49:10.198Z
- SHA-256 of the archived text
e8279a73f66591a6cfc060442fba29b88e5caf8b2653d6c7853c63d0fb7c3ac9- Stable URL of this page
https://metraipolis.com/documento/dc_2963541f4346
How to cite
Where A Small Language Model Helps in Invoice Categorisation, Understood Through Embedding Geometry — arXiv:2608.18033 — arXiv (en), https://arxiv.org/abs/2608.18033 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2963541f4346 (Metraipolis archive dc_2963541f4346, sha256 e8279a73f66591a6cfc060442fba29b88e5caf8b2653d6c7853c63d0fb7c3ac9)
BibTeX
@misc{dc2963541f4346,
title = {Where A Small Language Model Helps in Invoice Categorisation, Understood Through Embedding Geometry — arXiv:2608.18033},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.18033},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2963541f4346; sha256 e8279a73f66591a6cfc060442fba29b88e5caf8b2653d6c7853c63d0fb7c3ac9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18033. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Categorising invoices into the correct General Ledger (GL) code underpins financial reporting and tax compliance. This is a skilled accounting judgement rather than a routine task: the correct category depends subtly on the nature of the purchasing business, the vendor and the invoice text. Whilst AI is increasingly being adopted across industries to automate tasks, including invoice categorisation, implementations built on in-house small language models (SLMs) can simultaneously reduce cost and improve data security, confidentiality, and interpretability. We investigate this approach by first analysing the pre-trained embedding geometry of a small sentence transformer (SBERT) and classic SLM (DeBERTa). The sentence-embedding space of this financial corpus is globally anisotropic but composed of locally isotropic clusters, extending prior token-level findings to sentence embeddings in a financial setting, and these clusters are strongly correlated with the vendor identity. SBERT fine-tuned on a single GPU reaches 0.96 accuracy on invoice classification, above both a zero-shot LLM and a vendor identity baseline, increasing performance for smaller, challenging categories and new clients. For this important generalisation problem, SBERT reaches 0.9 F1 with roughly 100 client-specific invoices, showing that an in-house SLM implementation is promising. Combining these results with geometric analysis shows that pre-trained embedding geometry is associated with classification performance and reveals a counterintuitive finding that a structured input that would help a human reader does not improve the SLM performance. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.LG). Categorie arXiv: Machine Learning (stat.ML) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.18033 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T14:47:59.479Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).