Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models

Generation of plausible but incorrect factual information, often termed hallucination, has attracted significant research interest. Retrieval-augmented language model (RALM) – which enhances models with up-to-date knowledge – emerges as a promising method to reduce hallucination. However, existing RALMs may instead exacerbate hallucination when retrieving lengthy contexts. To address this challenge, we propose COFT, a novel $\mathbf{\text{CO}}$arse-to-$\mathbf{\text{F}}$ine highligh$\mathbf{\text{T}}$ing method to focus on different granularity-level key texts, thereby avoiding getting lost in lengthy contexts. Specifically, COFT consists of three components: $\mathit{\text{recaller}}$, $\mathit{\text{scorer}}$, and $\mathit{\text{selector}}$. First, $\mathit{\text{recaller}}$ applies a knowledge graph to extract potential key entities in a given context. Second, $\mathit{\text{scorer}}$ measures the importance of each entity by calculating its contextual weight. Finally, $\mathit{\text{selector}}$ selects high contextual weight entities with a dynamic threshold algorithm and highlights the corresponding paragraphs, sentences, or words in a coarse-to-fine manner. Extensive experiments on the knowledge hallucination benchmark demonstrate the effectiveness of COFT, leading to a superior performance over $30\mathrm{\%}$ in the F1 score metric. Moreover, COFT also exhibits remarkable versatility across various long-form tasks, such as reading comprehension and question answering.