Metraipolis › archive › dc_596e198a56f9
pico-type: A 1.5M-Parameter Byte-Level Multi-Head Content Classifier — arXiv:2608.14658
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14658
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T17:51:52.490Z
- SHA-256 of the archived text
81b9f8863f9bebb3fbd706a9b332f629462e3ba22f2de9cf60d8f4be5264e4bc- Stable URL of this page
https://metraipolis.com/documento/dc_596e198a56f9
How to cite
pico-type: A 1.5M-Parameter Byte-Level Multi-Head Content Classifier — arXiv:2608.14658 — arXiv (en), https://arxiv.org/abs/2608.14658 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_596e198a56f9 (Metraipolis archive dc_596e198a56f9, sha256 81b9f8863f9bebb3fbd706a9b332f629462e3ba22f2de9cf60d8f4be5264e4bc)
BibTeX
@misc{dc596e198a56f9,
title = {pico-type: A 1.5M-Parameter Byte-Level Multi-Head Content Classifier — arXiv:2608.14658},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14658},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_596e198a56f9; sha256 81b9f8863f9bebb3fbd706a9b332f629462e3ba22f2de9cf60d8f4be5264e4bc},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14658. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We introduce pico-type, a byte-level multi-head content classifier with approximately 1.5 million parameters that simultaneously predicts seven content properties from raw UTF-8 bytes in a single forward pass. Operating directly at the byte level -- no tokenizer, no subword vocabulary, no pretrained embeddings -- pico-type classifies coarse type (12 classes), modality (8), subtype (24), code language (62), text language (30), file MIME type (90), and risk flags (6-label multi-label: API keys, JWTs, passwords, emails, phone numbers, SSH keys). The architecture combines a learned byte embedding, three convolutional blocks with growing receptive fields, two bidirectional attention layers with rotary position encodings, and a statistical pooling layer feeding seven Matryoshka-style classification heads. Four tiered variants (tiny/small/base/pro) share the same trunk with sliced representations from 16 to 576 dimensions, yielding ONNX exports under 210 KB and CPU inference under 10 ms. Trained on a mixture of synthetic templates and real-world data (8709 GitHub code samples, 5000 Wikipedia articles), pico-type achieves 60.3 percent code language accuracy on The Heap benchmark (24 languages) and 98.2 percent text language accuracy on Wikipedia (30 languages) -- improvements of +57 and +79 percentage points respectively over the synthetic-only baseline. Format-based heads (coarse, modality, subtype, file_mime, risk) maintain 100 percent accuracy on synthetic benchmarks. The model, code, and pretrained weights are released under Apache 2.0. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.AI). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2608.14658 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T16:29:13.933Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).