RAGap: Detecting Hallucinations by Constructing Mechanistic Conflicts with Context Heads

摘要

Retrieval-augmented generation (RAG) aims to enhance factuality in Large Language Models (LLMs), yet these models frequently generate unfaithful outputs that contradict or exceed the provided context. While the internal representations of the LLM often contain features sufficient to flag hallucinations, the additive nature of the residual stream allows ingrained parametric knowledge to progressively outweigh contextual information during inference. In this work, we formalize this internal conflict as Knowledge-Context Gap (KCG). We first demonstrate that KCG serves as a robust indicator of faithfulness, though its characterization via prompt-level manipulation is hindered by computational inefficiency and representational drift. To address these challenges, we propose RAGap, a mechanistic framework that characterizes KCG by isolating the LLM's parametric knowledge directly within the residual stream. By identifying and suppressing specialized context heads through differentiable masking, RAGap enables an efficient characterization of the latent divergence between parametric knowledge and contextual information. Leveraging Sparse Autoencoders for interpretable feature extraction, our method consistently outperforms state-of-the-art hallucination detectors across multiple benchmarks. Our findings provide a novel mechanistic perspective on RAG faithfulness and offer a practical solution for reliable, inference-time hallucination detection.