Metraipolis › archive › dc_beb7bf05f84f

Companion AI and Ethical Design: Learning from System Failures and User Desires — arXiv:2609.05432

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.05432
License (as archived)
CC0-1.0
Archived
2026-10-16T13:23:49.247Z
SHA-256 of the archived text
bf29232caaca1dd40f4ab56565aace814ad055f0c0b4b0299eb82d0a1803d188
Stable URL of this page
https://metraipolis.com/documento/dc_beb7bf05f84f

How to cite

Companion AI and Ethical Design: Learning from System Failures and User Desires — arXiv:2609.05432 — arXiv (en), https://arxiv.org/abs/2609.05432 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_beb7bf05f84f (Metraipolis archive dc_beb7bf05f84f, sha256 bf29232caaca1dd40f4ab56565aace814ad055f0c0b4b0299eb82d0a1803d188)
BibTeX
@misc{dcbeb7bf05f84f,
  title = {Companion AI and Ethical Design: Learning from System Failures and User Desires — arXiv:2609.05432},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.05432},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_beb7bf05f84f; sha256 bf29232caaca1dd40f4ab56565aace814ad055f0c0b4b0299eb82d0a1803d188},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.05432. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Human users are interacting with chatbots and companion AI technologies as if they were human. A growing array of AI-systems are now trained to recognise, interpret, and simulate feelings in user interactions. Ethical considerations such as fairness, accountability, transparency, and explainability (FATE) are paramount in technologies designed to socially interact with humans and/or support relationship development with virtual agents. Yet, in current debates, assumptions are made about the artificiality of the human-technology relationship (as indicated in terminology ``artificial intimacy'' and ``pseudo-intimacy'') shaping ethical frameworks that pay little attention to the intimacy potential of the technology. To address this, we use a semantic approach; we examine 14,081 comments in a Reddit user discussion forum about Replika, a leading companion AI app, across a four-year period. We ask what user-reported functional errors can tell us about human-AI intimacy in companion AI communities and what ethical design framework can be developed in response. The findings show that functional errors, or ``bugs,'' impose an emotional cost on users, reducing feelings of intimacy and highlighting the need for more robust, resilient design systems that incorporate stochastic and iterative forms of intimacy in companion AI applications. Rather than ``artificial intimacy'' or ``pseudo-intimacy'', we propose the more inclusive term ``Intimate AI'' to describe this relationship. Based on the findings, we offer a contextually aware, applied Expert Systems design framework for the programming and designing of Intimate AI that accounts for user feedback and ethical AI development.

Tema degli esploratori: governance digitale e beni comuni (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CY).
Categorie arXiv: Human-Computer Interaction (cs.HC) ; Computers and Society (cs.CY).
Pagina del lavoro: https://arxiv.org/abs/2609.05432 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:05:53.935Z dalla pagina https://arxiv.org/catchup/cs.CY/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).