Knowledge Fusion via Bidirectional Information Aggregation

Knowledge graphs (KGs) are the cornerstone of the semantic web, offering up-to-date representations of real-world entities and relations. Yet large language models (LLMs) remain largely static after pre-training, causing their internal knowledge to become outdated and limiting their utility in time-sensitive web applications. To bridge this gap between dynamic knowledge and static models, a prevalent approach is to enhance LLMs with KGs. However, prevailing methods typically rely on parameter-invasive fine-tuning, which risks catastrophic forgetting and often degrades LLMs’ general capabilities. Moreover, their static integration frameworks cannot keep pace with the continuous evolution of real-world KGs, hindering their deployment in dynamic web environments. To bridge this gap, we introduce KGA ($\underset{‾}{\mathit{\text{K}}}\mathit{\text{nowledge}}\underset{‾}{\mathit{\text{G}}}\mathit{\text{raph-guided}}\underset{‾}{\mathit{\text{A}}}\mathit{\text{ttention}}$), a novel framework that dynamically integrates external KGs into LLMs exclusively at inference-time without any parameter modification. Inspired by research on neuroscience, we rewire the self-attention module by innovatively introducing two synergistic pathways: a $\mathit{\text{bottom-up}}\mathit{\text{knowledge}}\mathit{\text{fusion}}$ pathway and a $\mathit{\text{top-down}}\mathit{\text{attention}}\mathit{\text{guidance}}$ pathway. The $\mathit{\text{bottom-up}}\mathit{\text{pathway}}$ dynamically integrates external knowledge into input representations via input-driven KG fusion, which is akin to the $\mathit{\text{stimulus-driven}}\mathit{\text{attention}}\mathit{\text{process}}$ in the human brain. Complementarily, the $\mathit{\text{top-down}}\mathit{\text{pathway}}$ aims to assess the contextual relevance of each triple through a $\mathit{\text{goal-directed}}\mathit{\text{verification}}\mathit{\text{process}}$, thereby suppressing task-irrelevant signals and amplifying knowledge-relevant patterns. By synergistically combining these two pathways, our method supports real-time knowledge fusion. Extensive experiments on four benchmarks verify KGA’s strong fusion performance and efficiency.