LegoNet: Memory Footprint Reduction Through Block Weight Clustering

As the need for neural network-based applications to become more accurate and powerful grows, so too does their size and memory footprint. With embedded devices, whose cache and RAM are limited, this growth hinders their ability to leverage state-of-the-art neural network architectures. In this work, we propose $\mathbf{\text{LegoNet}}$, a compression technique that $\mathbf{\text{constructs}}\mathbf{\text{blocks}}\mathbf{\text{of}}\mathbf{\text{weights}}\mathbf{\text{of}}\mathbf{\text{the}}\mathbf{\text{entire}}\mathbf{\text{model}}\mathbf{\text{regardless}}\mathbf{\text{of}}\mathbf{\text{layer}}\mathbf{\text{type}}$ and clusters these induced blocks. Using blocks instead of individual values to cluster the weights, we were able to compress ResNet-50 trained for Cifar-10 and ImageNet with only 32 4x4 blocks, compressing the memory footprint by over a factor of $\mathbf{\text{64x}}\mathbf{\text{without}}\mathbf{\text{having}}\mathbf{\text{to}}\mathbf{\text{remove}}\mathbf{\text{any}}\mathbf{\text{weights}}$ or changing the architecture and $\mathbf{\text{no}}\mathbf{\text{loss}}\mathbf{\text{to}}\mathbf{\text{accuracy}}$, nor retraining or any data, and show how to find an arrangement of 16 4x4 blocks that gives a compression ratio of $\mathbf{\text{128x}}\mathbf{\text{with}}\mathbf{\text{less}}\mathbf{\text{than}}\mathbf{\text{3\%}}\mathbf{\text{accuracy}}\mathbf{\text{loss}}$. This was all achieved with $\mathbf{\text{no}}\mathbf{\text{need}}\mathbf{\text{for}}\mathbf{\text{(re)training}}\mathbf{\text{or}}\mathbf{\text{fine-tuning}}$.