Metraipolis › archive › dc_2f8d48eb868a

How Far Can Sub-3B Open Language Models Go in Zero-Shot Essay Scoring on an 8 GB Consumer GPU? — arXiv:2609.20250

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.20250
License (as archived)
CC0-1.0
Archived
2026-10-10T04:08:57.793Z
SHA-256 of the archived text
7e3f4580c085bb7774edd7dc5feafaa272d0d7c7f9e733c982c77200d38c961a
Stable URL of this page
https://metraipolis.com/documento/dc_2f8d48eb868a

How to cite

How Far Can Sub-3B Open Language Models Go in Zero-Shot Essay Scoring on an 8 GB Consumer GPU? — arXiv:2609.20250 — arXiv (en), https://arxiv.org/abs/2609.20250 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2f8d48eb868a (Metraipolis archive dc_2f8d48eb868a, sha256 7e3f4580c085bb7774edd7dc5feafaa272d0d7c7f9e733c982c77200d38c961a)
BibTeX
@misc{dc2f8d48eb868a,
  title = {How Far Can Sub-3B Open Language Models Go in Zero-Shot Essay Scoring on an 8 GB Consumer GPU? — arXiv:2609.20250},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.20250},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2f8d48eb868a; sha256 7e3f4580c085bb7774edd7dc5feafaa272d0d7c7f9e733c982c77200d38c961a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20250. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Zero-shot essay scoring with large language models is usually demonstrated with proprietary API models, yet the settings where automated scoring is most needed, such as public schools grading thousands of essays under strict privacy rules, are often those where sending student writing to a third-party API is unacceptable. We ask how much capability survives when the model must be a sub-3B open model running fully locally in FP16, with a controlled study of four instruction-tuned models from two families (Qwen2.5 at 0.5B/1.5B/3B, SmolLM2 at 1.7B) on all eight ASAP-AES prompts on a single 8 GB consumer GPU, with bootstrap confidence intervals, Holm-corrected paired tests, and deployment-realistic variants of the key design choices. Three findings emerge. (i) Rubric-decomposed prompting beats holistic prompting for every model under batch min-max aggregation (though Qwen2.5-3B drops significantly on one prompt), and under mean aggregation two unrelated families land within 0.01 at the 1.5-1.7B scale. (ii) Mapping trait scores into the prompt range is fragile to grader calibration: one model compresses traits into a narrow low band (2-4 on 0-10) and naive mean aggregation collapses, while the min-max normalization of Multi-Trait Specialization repairs it (macro QWK 0.204 to 0.388) and stays within 0.03 when its statistics are frozen on 30 held-out essays. (iii) Signed error falls with essay length in eleven of twelve configurations, opposite to the verbosity bias reported for large LLM judges; normalized rubric decomposition largely flattens this slope for well-calibrated models. We anchor results honestly: the best local configuration (0.388) remains far below both the human inter-rater ceiling (0.769) and a length-only baseline (0.523), so we position sub-3B local models strictly for formative, human-supervised feedback.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.20250 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:48:18.341Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).