Metraipolis › archive › dc_90c689a8e558
Audiovisual joint learning for end-to-end hearing aids — arXiv:2610.08579
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.08579
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T21:22:46.521Z
- SHA-256 of the archived text
8343ce443a3ae7fd57cfe18fb3b7a17804823c91446d44a2c07c3b3f378854b9- Stable URL of this page
https://metraipolis.com/documento/dc_90c689a8e558
How to cite
Audiovisual joint learning for end-to-end hearing aids — arXiv:2610.08579 — arXiv (en), https://arxiv.org/abs/2610.08579 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_90c689a8e558 (Metraipolis archive dc_90c689a8e558, sha256 8343ce443a3ae7fd57cfe18fb3b7a17804823c91446d44a2c07c3b3f378854b9)
BibTeX
@misc{dc90c689a8e558,
title = {Audiovisual joint learning for end-to-end hearing aids — arXiv:2610.08579},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.08579},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_90c689a8e558; sha256 8343ce443a3ae7fd57cfe18fb3b7a17804823c91446d44a2c07c3b3f378854b9},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08579. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Speech understanding in noise remains challenging for hearing-aid users, particularly in the presence of competing speakers. Conventional hearing aids typically perform speech enhancement (SE) and hearing-loss compensation in separate stages, which may cause enhancement errors and signal distortions to carry over to the amplification stage. Moreover, audio-only SE often provides limited benefits under competing-speech conditions because the target and interfering speech share similar acoustic characteristics, making them difficult to separate. To address these limitations, we propose AV-NeuroAMP, an end-to-end audiovisual framework that integrates noisy speech, target-talker video, and the listener's audiogram to jointly perform SE, personalized amplification, and dynamic-range compression. We further introduce audiogram-conditioned feature-wise linear modulation (AC-FiLM) to effectively incorporate listener-specific hearing profiles. Objective evaluations showed that AV-NeuroAMP outperformed conventional amplification, the audio-only NeuroAMP model, and two-stage systems on an in-domain English test set, and that these improvements were retained on an unseen Mandarin test set. Listening tests involving normal-hearing participants under simulated hearing loss and listeners with hearing loss further demonstrated improvements in speech quality and intelligibility, with the greatest benefits observed under competing-speech conditions. These findings support end-to-end audiovisual personalized amplification as a promising approach for improving hearing-aid performance in challenging acoustic environments. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv eess.AS, nuovi lavori del giorno). Categorie arXiv: Audio and Speech Processing (eess.AS). Pagina del lavoro: https://arxiv.org/abs/2610.08579 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:54:04.259Z dalla pagina https://arxiv.org/list/eess.AS/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).