5W1H Extraction With Large Language Models

The extraction of essential news elements through the 5W1H framework ($\mathit{\text{What}}$, $\mathit{\text{When}}$, $\mathit{\text{Where}}$, $\mathit{\text{Why}}$, $\mathit{\text{Who}}$, and $\mathit{\text{How}}$) is critical for event extraction and text summarization. The advent of Large language models (LLMs) such as ChatGPT presents an opportunity to address language-related tasks through simple prompts without fine-tuning models with much time. While ChatGPT has encountered challenges in processing longer news texts and analyzing specific attributes in context, especially answering questions about $\mathit{\text{What}}$, $\mathit{\text{Why}}$, and $\mathit{\text{How}}$. The effectiveness of extraction tasks is notably dependent on high-quality human-annotated datasets. However, the absence of such datasets for the 5W1H extraction increases the difficulty of fine-tuning strategies based on open-source LLMs. To address these limitations, first, we annotate a high-quality 5W1H dataset based on four typical news corpora ($\mathit{\text{CNN/DailyMail}}$, $\mathit{\text{XSum}}$, $\mathit{\text{NYT}}$, $\mathit{\text{RA-MDS}}$); second, we design several strategies from zero-shot/few-shot prompting to efficient fine-tuning to conduct 5W1H aspects extraction from the original news documents. The experimental results demonstrate that the performance of the fine-tuned models on our labelled dataset is superior to the performance of ChatGPT. Furthermore, we also explore the domain adaptation capability by testing the source-domain (e.g. NYT) models on the target domain corpus (e.g. CNN/DailyMail) for the task of 5W1H extraction.