Assessing Answer Accuracy, Hallucination, and Document Relevance in a RAG-Based Chatbot at Osnabrück University
Wurch, Marvin Ives René · osnaDocs (Osnabrück University) · 2025
Not cached yet. This record was fetched live for this request and is stored after the lookup. Reopening the page should serve the cached copy.
osnaDocs (Osnabrück University) · Osnabrück University · book · 2025
Abstract
This study evaluates a retrieval-augmented generation chatbot tailored for Osnabrück University, focusing on answer accuracy, hallucination, and the relevance of retrieved documents. Through human and automated evaluations conducted bilingually (German and English), it examines the chatbot's ability to deliver coherent, accurate, and contextually grounded responses to real user inquiries. Results highlight the chatbot's strengths in linguistic fluency and low hallucination rates but indicate variability in accuracy and context relevance.
The study underscores the importance of hybrid evaluation methods combining automated metrics and targeted human assessments, offering insights into future refinement of domain-specific chatbots.
Open the resolved locator · 10.48693/655
Metadata licence: OpenAlex metadata (CC0)