Metraipolis › archive › dc_6396b1cd7b2b

On the Capability and Limitation of Hard Prompt — arXiv:2609.32302

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.32302
License (as archived)
CC0-1.0
Archived
2026-10-07T22:36:28.967Z
SHA-256 of the archived text
2c5fea26484cd3a23468db6e6c9e1b66e9704b562a9e65e10915e459aaf88107
Stable URL of this page
https://metraipolis.com/documento/dc_6396b1cd7b2b

How to cite

On the Capability and Limitation of Hard Prompt — arXiv:2609.32302 — arXiv (en), https://arxiv.org/abs/2609.32302 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6396b1cd7b2b (Metraipolis archive dc_6396b1cd7b2b, sha256 2c5fea26484cd3a23468db6e6c9e1b66e9704b562a9e65e10915e459aaf88107)
BibTeX
@misc{dc6396b1cd7b2b,
  title = {On the Capability and Limitation of Hard Prompt — arXiv:2609.32302},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.32302},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6396b1cd7b2b; sha256 2c5fea26484cd3a23468db6e6c9e1b66e9704b562a9e65e10915e459aaf88107},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.32302. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Prompt engineering has become an indispensable tool for using large language models (LLMs), turning LLMs into task-specific experts without changing their weights. Despite notable theoretical advances in prompt engineering, the theory for the more practical hard or discrete prompts is largely open. In this paper, we try to fill this gap either by providing a complete solution or by making substantial progress on the three core theoretical questions regarding hard prompts. First, we show that determining the existence of a hard prompt for a transformer to solve a downstream task is NP-complete and that finding an optimal hard prompt is NP-hard, which is the first computational complexity result for hard prompting, as far as we know. Second, we show that, unlike soft or continuous prompts, hard prompts have essential limitations: hard prompts are not complete; short hard prompts do not significantly enhance the ability of transformers; and long hard prompts exhibit the "prompt dominating answer phenomenon," meaning that, with high probability, the same answer is given for all queries of the same length. On the other hand, linear hard prompts do not have the limitations of short or long prompts. Third, we provide a tight bound on the size of the task in terms of the prompt length for the performance of prompts on the finite task to generalize to the entire data distribution, leading to a necessary and sufficient condition for generalizability. This is the first result on generalization for prompting, as far as we know. Our findings not only offer the first theoretical insights into hard prompts but also provide provably reliable practical guidance for real-world LLM usage.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.32302 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T22:14:27.653Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).