Metraipolis › archive › dc_ed945830fa41

A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model — arXiv:2611.00783

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00783
License (as archived)
CC0-1.0
Archived
2026-11-03T10:31:29.990Z
SHA-256 of the archived text
5ee5ab878f0843c064ea769e5740354e9d5315e61846ba47102d23af95333346
Stable URL of this page
https://metraipolis.com/documento/dc_ed945830fa41

How to cite

A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model — arXiv:2611.00783 — arXiv (en), https://arxiv.org/abs/2611.00783 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ed945830fa41 (Metraipolis archive dc_ed945830fa41, sha256 5ee5ab878f0843c064ea769e5740354e9d5315e61846ba47102d23af95333346)
BibTeX
@misc{dced945830fa41,
  title = {A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model — arXiv:2611.00783},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00783},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ed945830fa41; sha256 5ee5ab878f0843c064ea769e5740354e9d5315e61846ba47102d23af95333346},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00783. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Calling a tool through generated text adds decoding steps, even when the computation itself is short. We study an alternative: a small neural module that runs inside a language model's forward pass. A learned writer turns hidden states into a symbolic tape, a short sequence of discrete symbols that encodes the computation. The module, trained separately and then frozen, reduces the tape to a result, and a learned reader returns that result to the model. We evaluate arithmetic and object tracking in gemma-3-1b-it. With a writer trained on labelled tapes, the layer answers all 750 items of BIG-Bench Hard's tracking shuffled objects correctly. At seven people it needs about 1/10 of the GPU kernel time of a text-tool baseline that uses the same host model and exact code, with accuracy 1.00 against 0.99. The writer can also be learned from answers alone, guided by a hand-written copying prior, but less reliably: 5 of 6 runs succeed in the 1B host and 2 of 4 in the 4B host. Two independently trained layers keep their task accuracy when attached to one model, and a learned gate keeps them silent on the tested general-text and look-alike prompts. The results support a narrow use case: a bounded computation that returns one symbol can avoid the decoding cost of a textual tool call. Extracting the right input reliably, and deciding when to act, remain the main open problems.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Neural and Evolutionary Computing (cs.NE).
Pagina del lavoro: https://arxiv.org/abs/2611.00783 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:29:48.478Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).