ConvFormer: Parameter Reduction in Transformer Models for 3D Human Pose Estimation by Leveraging Dynamic Multi-Headed Convolutional Attention

Recently, fully-transformer architectures have replaced the defacto convolutional architecture for the 3D human pose estimation task. In this paper we propose $\mathit{\text{ConvFormer}}$, a novel convolutional transformer that leverages a new $\mathit{\text{dynamic}}\mathit{\text{multi-headed}}\mathit{\text{convolutional}}\mathit{\text{self-attention}}$ mechanism for monocular 3D human pose estimation. We designed a spatial and temporal convolutional transformer to comprehensively model human joint relations within individual frames and globally across the motion sequence. Moreover, we introduce a novel notion of $\mathit{\text{temporal}}\mathit{\text{joints}}\mathit{\text{profile}}$ for our temporal ConvFormer that fuses complete temporal information immediately for a local neighborhood of joint features. We have quantitatively and qualitatively validated our method on three common benchmark datasets: Human3.6M, MPI-INF-3DHP, and HumanEva. Extensive experiments have been conducted to identify the optimal hyper-parameter set. These experiments demonstrated that we achieved a $\mathbf{\text{significant}}\mathbf{\text{parameter}}\mathbf{\text{reduction}}\mathbf{\text{relative}}\mathbf{\text{to}}\mathbf{\text{prior}}\mathbf{\text{transformer}}\mathbf{\text{models}}$ while attaining State-of-the-Art (SOTA) or near SOTA on all three datasets. Additionally, we achieved SOTA for Protocol III on H36M for both GT and CPN detection inputs. Finally, we obtained SOTA on all three metrics for the MPI-INF-3DHP dataset and for all three subjects on HumanEva under Protocol II.