GUITester: Enabling GUI Agents for Exploratory Defect Discovery

Exploratory GUI testing is essential for software quality but suffers from high manual costs. While Multi-modal Large Language Model (MLLM) agents excel in navigation, they fail to autonomously discover defects due to two core challenges: $\mathit{\text{Goal-Oriented}}\mathit{\text{Masking}}$, where agents prioritize task completion over reporting anomalies, and $\mathit{\text{Execution-Bias}}\mathit{\text{Attribution}}$, where system defects are misidentified as agent errors. To address these, we first introduce $\mathbf{\text{GUITestBench}}$, the first interactive benchmark for this task, featuring 143 tasks across 26 defects. We then propose $\mathbf{\text{GUITester}}$, a multi-agent framework that decouples navigation from verification via two modules: (i) a $\mathit{\text{Planning-Execution}}\mathit{\text{Module}}\mathit{\text{(PEM)}}$ that proactively probes for defects via embedded testing intents, and (ii) a $\mathit{\text{Hierarchical}}\mathit{\text{Reflection}}\mathit{\text{Module}}\mathit{\text{(HRM)}}$ that resolves attribution ambiguity through interaction history analysis. GUITester achieves an F1-score of 48.90% (Pass@3) on GUITestBench, outperforming state-of-the-art baselines (33.35%). Our work demonstrates the feasibility of autonomous exploratory testing and provides a robust foundation for future GUI quality assurance~\footnote{Our code is now available in~\href{https://github.com/ADaM-BJTU/GUITestBench}{https://github.com/ADaM-BJTU/GUITestBench}}.