Metraipolis › archive › dc_34e30ab652df

Learning from imperfect teachers for low-resource acoustic generalization — arXiv:2610.05256

Source: arXiv
Original: https://arxiv.org/abs/2610.05256
License (as archived): CC0-1.0
Archived: 2026-10-06T05:25:05.190Z
SHA-256 of the archived text: 6f02778974318c5ae776df214fdc268348d9eb89520303494ffd8b836c72caba
Stable URL of this page: https://metraipolis.com/documento/dc_34e30ab652df

How to cite

Learning from imperfect teachers for low-resource acoustic generalization — arXiv:2610.05256 — arXiv, https://arxiv.org/abs/2610.05256 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_34e30ab652df (Metraipolis archive dc_34e30ab652df, sha256 6f02778974318c5ae776df214fdc268348d9eb89520303494ffd8b836c72caba)

BibTeX

@misc{dc34e30ab652df,
  title = {Learning from imperfect teachers for low-resource acoustic generalization — arXiv:2610.05256},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.05256},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_34e30ab652df; sha256 6f02778974318c5ae776df214fdc268348d9eb89520303494ffd8b836c72caba},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.05256. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Knowledge distillation (KD) improves low-resource acoustic learning by enriching one-hot supervision with the softened predictive distribution of a fixed teacher network. However, a teacher trained with limited or imbalanced annotations may produce a biased distribution whose components are not uniformly reliable. Although this distribution can still encode useful knowledge, direct full-distribution matching may also transfer teacher-induced biases, thereby distorting the student's decision boundary and degrading its generalization performance. To address this limitation, we propose Boundary-Anchored Mass-Partitioned Distillation (BA-MPD), a logit-based distillation objective composed of Boundary-Anchored Correction (BAC) and Mass-Partitioned Distillation (MPD). BAC addresses missing ground-truth labels in the set of the teacher's top predictions by swapping the true label for the lowest-ranked entry of the set, thus keeping the mass and uncertainty of the set unchanged. MPD then distills this corrected distribution through separate losses that enforce relational consistency within the set, balance the mass between high- and low-confidence groups, and weight lower-confidence dependencies. Ultimately, BAC and MPD together suppress harmful ranking errors and noisy low-confidence details, while retaining all useful teacher information. Experiments on two acoustic benchmarks under multiple label budgets show that BA-MPD consistently improves over supervised-learning baselines and vanilla KD while remaining competitive with strong logit-based KD baselines. Cross-budget results further show that BA-MPD remains effective when the teacher and student models use mismatched label budgets, demonstrating its ability to exploit imperfect teachers across supervision gaps. Implementation available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.05256 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-06T05:24:05.569Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).