Metraipolis › archive › dc_7366a16ffc36

EvalConvoLearn: An Open-Source Framework for Evaluating Grounded Learner Simulations in Tutoring Conversations — arXiv:2608.07497

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.07497
License (as archived)
CC0-1.0
Archived
2026-10-19T08:50:34.451Z
SHA-256 of the archived text
1b9b8669995df8e5dacbb6b9fa2f2229c009ae6b42fae50421cd3d14984a5144
Stable URL of this page
https://metraipolis.com/documento/dc_7366a16ffc36

How to cite

EvalConvoLearn: An Open-Source Framework for Evaluating Grounded Learner Simulations in Tutoring Conversations — arXiv:2608.07497 — arXiv (en), https://arxiv.org/abs/2608.07497 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7366a16ffc36 (Metraipolis archive dc_7366a16ffc36, sha256 1b9b8669995df8e5dacbb6b9fa2f2229c009ae6b42fae50421cd3d14984a5144)
BibTeX
@misc{dc7366a16ffc36,
  title = {EvalConvoLearn: An Open-Source Framework for Evaluating Grounded Learner Simulations in Tutoring Conversations — arXiv:2608.07497},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.07497},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7366a16ffc36; sha256 1b9b8669995df8e5dacbb6b9fa2f2229c009ae6b42fae50421cd3d14984a5144},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07497. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Conversational learner simulations are valuable tools for testing learning theories, evaluating instructional materials and automated tutors, or powering teachable agents. Recently, large language models (LLM) have enabled richer, more naturalistic interactions with simulated learners; however, no open framework exists for evaluating whether such simulations faithfully reproduce real learner behavior. We introduce EvalConvoLearn, an open-source framework that assesses learner simulations along two axes: learning behavior (skill-conditioned mastery outcomes) and conversational quality (talk moves, error type distributions, question rate, turn length). EvalConvoLearn measures how closely a simulated learner approximates answer distributions observed in data by grounding metrics in authentic tutoring conversation datasets, and anchoring generated tutor responses in existing tutor utterances. The framework is demonstrated on a dataset of tutoring dialogues, including results for two LLM-based learner simulations, and the published GitHub code.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Human-Computer Interaction (cs.HC) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.07497 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.953Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).