Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Drift

This paper studies $\mathbf{\text{thinking–answer}}\mathbf{\text{consistency}}$ in vision-language models. We focus on Visual Intention Grounding, where a model infers a target object based on a human intention query and predicts a bounding box. We reveal that previous IoU-based reinforcement learning (RL) frameworks suffer from ``thinking drift’', where the model produces a correct bounding box, despite having an incorrect reasoning process pointing to a different target object. Thus, we propose $\mathbf{\text{Rita}}$ ($\mathit{\text{ReInforcing}}\mathit{\text{Thinking–Answer}}\mathit{\text{consistency}}$) as a novel RL paradigm to tame the drift. Specifically, Rita introduces two reasoning-label-free RL rewards, constructed from the conditional probability of reference answers: a $\mathbf{\text{thinking}}\mathbf{\text{reward}}$ and a $\mathbf{\text{consistency}}\mathbf{\text{reward}}$. It also adopts a difficulty-aware $\mathbf{\text{data}}\mathbf{\text{filtering}}$ strategy that selects informative easy-to-medium samples for RL using rollout error rate and reward variance. Extensive experiments on EgoIntention and the new RefEgo-Int benchmarks show that Rita performs consistently superior to the supervised finetuning approaches and vanilla RL-finetuned frameworks.