Can the Environment Speak for Itself? $T^2$-GRPO: A Turn-Trajectory Group Relative Policy Optimization for Caregiver Agents

Optimizing large language models (LLMs) for long-horizon caregiver agents requires balancing delayed task objectives with immediate environment dynamics, such as patient distress and resistance. In dementia care, this balance is especially difficult: trajectory level rewards are too sparse for turn level credit assignment, while external LLM-based evaluators are costly and can misread fragmented or indirect patient responses. To address this issue, we propose $\mathbf{\text{T}}$urn-$\mathbf{\text{T}}$rajectory $\mathbf{\text{G}}$roup $\mathbf{\text{R}}$elative $\mathbf{\text{P}}$olicy $\mathbf{\text{O}}$ptimization ($\mathbf{\text{T}}{}^2\mathbf{\text{-GRPO}}$), a framework that decouples caregiver RL into two normalized reward horizons and enforces safety through a binary hard veto. $T^2$-GRPO derives dense turn-level rewards directly from environment state transitions, measuring changes in patient distress and resistance from a frozen dementia patient simulator. These environment-grounded rewards are combined with trajectory-level evaluations through independent centered-rank normalization, which preserves heterogeneous reward signals and mitigates reward collapse. Extensive experiments on dementia caregivers show that T ${}^2$-GRPO outperforms competitive baselines, indicating a substantial improvement for emotionally sensitive caregiver scenarios that effectively handles immediate patient feedback, long-term care outcomes, and safety constraints.