Metraipolis › archive › dc_55b5127f2fed

Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation — arXiv:2608.21229

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.21229
License (as archived)
CC0-1.0
Archived
2026-10-18T10:56:25.221Z
SHA-256 of the archived text
9369fed70185bb33e8db147e8854ba5ca63035f7cc1b5037daafd5d6ea3938cb
Stable URL of this page
https://metraipolis.com/documento/dc_55b5127f2fed

How to cite

Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation — arXiv:2608.21229 — arXiv (en), https://arxiv.org/abs/2608.21229 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_55b5127f2fed (Metraipolis archive dc_55b5127f2fed, sha256 9369fed70185bb33e8db147e8854ba5ca63035f7cc1b5037daafd5d6ea3938cb)
BibTeX
@misc{dc55b5127f2fed,
  title = {Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generation — arXiv:2608.21229},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.21229},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_55b5127f2fed; sha256 9369fed70185bb33e8db147e8854ba5ca63035f7cc1b5037daafd5d6ea3938cb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.21229. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In-context diffusion transformers concatenate instruction, target, and reference tokens into a single sequence for joint attention. Reference-side computation must therefore be repeated at every denoising step, with the cost growing rapidly as more references are added. Decoupling reference tokens from the target enables exact key-value reuse across denoising steps, but prevents the references from attending to the instruction, degrading instruction following and reference fidelity. This trade-off cannot be resolved through attention-mask design alone. We introduce AnchorCache, a parameter-free token-layout and attention-mask co-design that inserts static text anchors. These anchors condition the reference representations on the instruction during cache construction, after which the resulting reference keys and values can be reused exactly across denoising steps. To recover the quality initially lost through this structural conversion, we apply teacher-forced velocity distillation followed by a short on-policy stage that queries the teacher at student-visited states. To our knowledge, this is the first use of on-policy distillation for architectural recovery in diffusion models. Across benchmarks spanning image, speech, and video generation, AnchorCache matches full-attention quality. Its efficiency gains increase with the reference-context size, reaching a 6.40x speedup in diffusion transformer inference.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-24 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.21229 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:48:13.604Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).