MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark

Machine-Generated Text (MGT) is becoming increasingly difficult to distinguish from Human-Written Text (HWT). This trend has exacerbated malicious activities such as fake news and online fraud. The generalization ability of fine-tuned detectors relies heavily on dataset quality, and simply expanding the sources of MGT may become increasingly insufficient. Further augmentation of the generation process is required. Based on HC-Var’s theory, enhancing the human-like alignment of MGT not only facilitates robustness testing of existing detectors but also boosts the generalization ability of detectors fine-tuned on such aligned MGT datasets. Therefore, we propose the $\mathbf{\text{M}}$achine-$\mathbf{\text{A}}$ugment-$\mathbf{\text{G}}$enerated Text via $\mathbf{\text{A}}$lignment (MAGA) Detection Benchmark. MAGA integrates several alignment methods, ranging from prompt construction to $\mathbf{\text{G}}$enerator-$\mathbf{\text{D}}$etector $\mathbf{\text{A}}$dversarial $\mathbf{\text{R}}$einforcement $\mathbf{\text{L}}$earning (GDARL) and the reasoning process. In our experiments, the RoBERTa detector fine-tuned on MAGA achieves an average improvement of 4.60% in generalization AUC. Conversely, the aligned MGTs in MAGA also lead to an average decrease of 8.13% in the AUC of selected detectors. We hope the MAGA Benchmark will provide valuable insights for future research on the generalization ability of MGT detectors.