Metraipolis › archive › dc_d2b787ce2e54

LINE Conversation History Retrieval for Personal Memory RAG: Evaluating Search Representations and Hybrid Retrieval — arXiv:2608.27809

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.27809
License (as archived)
CC0-1.0
Archived
2026-10-17T11:28:18.207Z
SHA-256 of the archived text
0cb9875298f2a0c3209588f3c800011c08d78b3f5f8e7d3a760086e05cf7105d
Stable URL of this page
https://metraipolis.com/documento/dc_d2b787ce2e54

How to cite

LINE Conversation History Retrieval for Personal Memory RAG: Evaluating Search Representations and Hybrid Retrieval — arXiv:2608.27809 — arXiv (en), https://arxiv.org/abs/2608.27809 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d2b787ce2e54 (Metraipolis archive dc_d2b787ce2e54, sha256 0cb9875298f2a0c3209588f3c800011c08d78b3f5f8e7d3a760086e05cf7105d)
BibTeX
@misc{dcd2b787ce2e54,
  title = {LINE Conversation History Retrieval for Personal Memory RAG: Evaluating Search Representations and Hybrid Retrieval — arXiv:2608.27809},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.27809},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d2b787ce2e54; sha256 0cb9875298f2a0c3209588f3c800011c08d78b3f5f8e7d3a760086e05cf7105d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.27809. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As an initial step toward personal memory retrieval-augmented generation (RAG) for large language models (LLMs), this study presents a retrieval-only case study over one user's LINE conversation history. We segmented 358,896 messages into 22,329 temporally coherent chunks and constructed three search representations: raw_text, a generated summary, and embedding_text, which combines a summary with a raw-text excerpt and other fixed text. We compared BM25, dense vector retrieval, and linear hybrid retrieval on 100 evaluation questions verified by a single annotator. Among individual retrievers, embedding_text_bm25 achieved the highest point estimate, with Recall@5 of 0.584. We then explored six retriever pairings and 21 weights, for 126 configurations on the same evaluation set. The selected combination of embedding_text_bm25 and embedding_text_vector at beta = 0.45 achieved Recall@5 = 0.697, MRR@5 = 0.595, and nDCG@5 = 0.575. Its Recall@5 exceeded that of embedding_text_bm25 by 0.113, with a question-level paired percentile-bootstrap 95% confidence interval of [0.048, 0.184]. This interval is conditional on fixing the configuration selected on the same 100 questions and does not account for uncertainty from configuration selection or weight search. The difference from a summary-based hybrid at beta = 0.50 was 0.050, with a 95% confidence interval of [-0.013, 0.115], so no clear difference could be established. The 17 aggregate questions also yielded lower point estimates than the other question types, suggesting that flat chunk-level retrieval struggles when evidence is distributed across multiple times and conversations. This evaluation is an exploratory single-user, single-annotator study conducted on the same question set used for configuration search; it does not evaluate final answer generation or generalization to unseen questions.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-31 in cs.IR).
Categorie arXiv: Information Retrieval (cs.IR).
Pagina del lavoro: https://arxiv.org/abs/2608.27809 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T11:09:46.448Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-08-31?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).