DROD: Dual-Ratio Ordered Dropout for Federated Continual Learning on Heterogeneous Devices

Federated learning for capacity-heterogeneous clients and federated continual learning are each well studied on their own, but their intersection – clients holding differently sized models while every client’s data stream also drifts over time – remains underexplored. We study this setting directly, training a ResNet-18 fleet from scratch on Split-CIFAR-100 (10 tasks, class-incremental), and introduce DROD (Dual-Ratio Ordered Dropout), which builds on FjORD’s Ordered Dropout by using two independent reduction axes to extract sub-models: a shortcut ratio and a branch ratio. On a shared parameter/FLOP budget ladder, DROD achieves the highest mean accuracy among the heterogeneous-model federated learning strategies we compare. Its advantage over the two strongest baseline geometries persists across every replay-buffer size we test and under ER, DER++, and BiC; under EWC, all three geometries collapse alike.