Metraipolis › archive › dc_d40fc01752ec

A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model — arXiv:2611.00783

Source: arXiv (en)
Original: https://arxiv.org/abs/2611.00783
License (as archived): CC0-1.0
Archived: 2026-11-03T11:01:30.236Z
SHA-256 of the archived text: 18b6bb941e84c3becd24a35ad359d1add725f23c761a506e912d76625b53a4d7
Stable URL of this page: https://metraipolis.com/documento/dc_d40fc01752ec

How to cite

A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model — arXiv:2611.00783 — arXiv (en), https://arxiv.org/abs/2611.00783 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d40fc01752ec (Metraipolis archive dc_d40fc01752ec, sha256 18b6bb941e84c3becd24a35ad359d1add725f23c761a506e912d76625b53a4d7)

BibTeX

@misc{dcd40fc01752ec,
  title = {A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model — arXiv:2611.00783},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00783},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d40fc01752ec; sha256 18b6bb941e84c3becd24a35ad359d1add725f23c761a506e912d76625b53a4d7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00783. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Calling a tool through generated text adds decoding steps, even when the computation itself is short. We study an alternative: a small neural module that runs inside a language model's forward pass. A learned writer turns hidden states into a symbolic tape, a short sequence of discrete symbols that encodes the computation. The module, trained separately and then frozen, reduces the tape to a result, and a learned reader returns that result to the model. We evaluate arithmetic and object tracking in gemma-3-1b-it. With a writer trained on labelled tapes, the layer answers all 750 items of BIG-Bench Hard's tracking shuffled objects correctly. At seven people it needs about 1/10 of the GPU kernel time of a text-tool baseline that uses the same host model and exact code, with accuracy 1.00 against 0.99. The writer can also be learned from answers alone, guided by a hand-written copying prior, but less reliably: 5 of 6 runs succeed in the 1B host and 2 of 4 in the 4B host. Two independently trained layers keep their task accuracy when attached to one model, and a learned gate keeps them silent on the tested general-text and look-alike prompts. The results support a narrow use case: a bounded computation that returns one symbol can avoid the decoding cost of a textual tool call. Extracting the right input reliably, and deciding when to act, remain the main open problems.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Neural and Evolutionary Computing (cs.NE).
Pagina del lavoro: https://arxiv.org/abs/2611.00783 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:59:45.742Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).