On Generating Explanations for Reinforcement Learning Policies: An Empirical Study

Understanding a $\mathit{\text{reinforcement}}\mathit{\text{learning}}$ policy, which guides state-to-action mappings to maximize rewards, necessitates an accompanying explanation for human comprehension. In this paper, we introduce a set of $\mathit{\text{linear}}\mathit{\text{temporal}}\mathit{\text{logic}}$ formulae designed to provide explanations for policies, and an algorithm for searching through those formulae for the one that best explains a given policy. Our focus is on explanations that elucidate both the ultimate objectives accomplished by the policy and the prerequisite conditions it upholds throughout its execution. The effectiveness of our proposed approach is illustrated through a simulated game of capture-the-flag and a car-parking environment,