Prior2Posterior: Model Prior Correction for Long-Tailed Learning

Learning-based solutions for long-tailed recognition face difficulties in generalizing on balanced test datasets. Due to imbalanced data prior, the learned $\mathit{\text{a}}\mathit{\text{posteriori}}$ distribution is biased toward the most frequent (head) classes, leading to an inferior performance on the least frequent (tail) classes. In general, the performance can be improved by removing such a bias by eliminating the effect of imbalanced prior modeled using the number of class samples (frequencies). We first observe that the $\mathit{\text{effective}}\mathit{\text{prior}}$ on the classes, learned by the model at the end of the training, can differ from the empirical prior obtained using class frequencies. Thus, we propose a novel approach to accurately model the effective prior of a trained model using $\mathit{\text{a}}\mathit{\text{posteriori}}$ probabilities. We propose to correct the imbalanced prior by adjusting the predicted $\mathit{\text{a}}\mathit{\text{posteriori}}$ probabilities (Prior2Posterior: P2P) using the calculated prior in a post-hoc manner after the training, and show that it can result in improved model performance. We present theoretical analysis showing the optimality of our approach for models trained with naive cross-entropy loss as well as logit adjusted loss. Our experiments show that the proposed approach achieves new state-of-the-art (SOTA) on several benchmark datasets from the long-tail literature in the category of logit adjustment methods. Further, the proposed approach can be used to inspect any existing method to capture the $\mathit{\text{effective}}\mathit{\text{prior}}$ and remove any residual bias to improve its performance, post-hoc, without model retraining. We also show that by using the proposed post-hoc approach, the performance of many existing methods can be improved further.