FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivative

This paper introduces $\mathtt{\text{FedMPDD}}$ ($\mathbf{\text{Fed}}$erated Learning via $\mathbf{\text{M}}$ulti-$\mathbf{\text{P}}$rojected $\mathbf{\text{D}}$irectional $\mathbf{\text{D}}$erivatives), a novel algorithm that simultaneously optimizes bandwidth utilization and enhances privacy in Federated Learning. The core idea of $\mathtt{\text{FedMPDD}}$ is to encode each client’s high-dimensional gradient by computing its directional derivatives along multiple random vectors. This compresses the gradient into a much smaller message, significantly reducing uplink communication costs from $\mathcal{O}(d)$ to $\mathcal{O}(m)$, where $m\ll d$. The server then decodes the aggregated information by projecting it back onto the same random vectors. Our key insight is that averaging multiple projections overcomes the dimension-dependent convergence limitations of a single projection. We provide a rigorous theoretical analysis, establishing that $\mathtt{\text{FedMPDD}}$ converges at a rate of $\mathcal{O}(1\mathrm{/}\sqrt{K})$, matching the performance of FedSGD. Furthermore, we demonstrate that our method provides some inherent privacy against gradient inversion attacks due to the geometric properties of low-rank projections, offering a tunable privacy-utility trade-off controlled by the number of projections. Extensive experiments on benchmark datasets validate our theory and demonstrates our results.