An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs

arXiv:2406.11290v3 Announce Type: replace-cross Abstract: Relevance and utility are two frequently used measures to evaluate the effectiveness of an information retrieval (IR) system. Relevance emphasizes the aboutness of a result to a query, while utility refers to the result's usefulness or value to an information seeker. In retrieval-augmented generation (RAG), high-utility results should be prioritized to feed to LLMs due to their limited input bandwidth. Re-examining RAG's three core compon