On the Time Derivative of the KL Divergence for a Generalized Langevin Annealing Scheme

Consider the Langevin diffusion process $\mathrm{d}{X}_t=\mathrm{\nabla }\log p_t(X_t)+\sqrt{2}\mathrm{d}{W}_t$ guided by the time-dependent probability density $p_t(x)$. Let $q_t$ be the density of $X_t$. Recently, in order to analyze convergence in the Kullback-Leibler divergence, the time derivative of $t\mapsto \mathrm{K}\mathrm{L}(q_t\mathrm{\mid }{p}_t)$ has been used in several works without investigating in detail when such a derivative exists. In this short manuscript we provide a rigorous derivation of the quantity $\frac{\mathrm{d}}{\mathrm{d}t}\mathrm{K}\mathrm{L}(q_t\mathrm{\mid }{p}_t)$.