Revisiting Copying Heads from a Spatiotemporal Perspective for Faithful Retrieval-Augmented Generation

摘要

Retrieval-Augmented Generation (RAG) may produce hallucinations even when the retrieved evidence is accurate and relevant, making faithful context utilization a key challenge. Prior studies have identified \emph{Copying Heads}, a group of attention heads that retrieve relevant contextual information and play a key role in contextual faithfulness. However, existing methods leverage copying heads as static and isolated components, overlooking their cross-head coordination within layers and temporal dynamics during autoregressive decoding. We address this gap by establishing a spatiotemporal analysis framework for copying heads. Spatially, we introduce \textit{Copying--Non-copying Attention Inconsistency} (CNC) to measure cross-head differences in contextual attention. Temporally, we introduce \textit{Copy Attention Ratio} (CAR) to track contextual reliance over decoding . We observe that higher CNC is associated with greater token-level hallucination risk, while copying heads exhibit stronger temporal decay during hallucinated generation. These findings motivate us to develop a mechanism-guided spatiotemporal regulation framework. It reduces excessive cross-head divergence and mitigates temporal decay in contextual reliance during decoding. Experiments demonstrate that our method improves contextual faithfulness, reduces cross-head inconsistency, and mitigates contextual-copying decay.