go-v.orgdoi

Assessing Answer Accuracy, Hallucination, and Document Relevance in a RAG-Based Chatbot at Osnabrück University

Wurch, Marvin Ives René · osnaDocs (Osnabrück University) · 2025

  • DOI 10.48693/655
  • Publisher Osnabrück University
  • Type book
  • Issued 2025

Not cached yet. This record was fetched live for this request and is stored after the lookup. Reopening the page should serve the cached copy.

osnaDocs (Osnabrück University) · Osnabrück University · book · 2025

Abstract

This study evaluates a retrieval-augmented generation chatbot tailored for Osnabrück University, focusing on answer accuracy, hallucination, and the relevance of retrieved documents. Through human and automated evaluations conducted bilingually (German and English), it examines the chatbot's ability to deliver coherent, accurate, and contextually grounded responses to real user inquiries. Results highlight the chatbot's strengths in linguistic fluency and low hallucination rates but indicate variability in accuracy and context relevance.

The study underscores the importance of hybrid evaluation methods combining automated metrics and targeted human assessments, offering insights into future refinement of domain-specific chatbots.

Open the resolved locator · 10.48693/655

Metadata licence: OpenAlex metadata (CC0)