Stealth Fine-Tuning: Efficiently Breaking Alignment in RVLMs Using Self-Generated CoT

Reasoning-augmented Vision-Language Models (RVLMs) rely on safety alignment to prevent harmful behavior, yet their exposed chain-of-thought (CoT) traces introduce new attack surfaces. In this work, we find that the safety alignment of RVLMs can be easily broken through a novel attack method termed $\mathbf{\text{Stealth}}\mathbf{\text{Fine-Tuning}}$. Our method elicits harmful reasoning traces through $\mathbf{\text{segment-level}}\mathbf{\text{interference}}$ and reuses the self-generated outputs as supervised fine-tuning data. To facilitate this, we introduce a $\mathbf{\text{turn-based}}\mathbf{\text{weighted}}$ loss that minimizes distribution shift. In our experiment, with only 499 samples and under 3 hours on a single A100 (QLoRA), Stealth Fine-Tuning outperforms IDEATOR by 38.66% ASR while preserving general reasoning ability, as the tuned model retains the original representation distribution. Experiments on AdvBench and several general benchmarks demonstrate that Stealth Fine-Tuning is a low-cost and highly effective way to bypass alignment defenses. \textcolor{red}{$\mathbf{\text{Disclaimer:}}\mathbf{\text{This}}\mathbf{\text{paper}}\mathbf{\text{contains}}\mathbf{\text{content}}\mathbf{\text{that}}\mathbf{\text{may}}\mathbf{\text{be}}\mathbf{\text{disturbing}}\mathbf{\text{or}}\mathbf{\text{offensive.}}$}