LipCache: A Local Inference Proxy with Certified Caching for Edge Image Classification Service

As edge-side vision services continue to expand toward low-latency, high-throughput scenarios, reducing the inference cost of vision models without sacrificing reliability has become a central concern. Existing semantic caching methods largely rely on empirical similarity thresholds; while such thresholds improve hit rates, they tend to introduce silent misclassifications near decision boundaries. To address this, we propose $\mathtt{\text{LipCache}}$, a certified semantic caching framework for image classification. Without modifying the existing deployed main model, $\mathtt{\text{MainNet}}$, the framework introduces a lightweight network, $\mathtt{\text{GuardNet}}$, that maps inputs into a low-dimensional feature space subject to a Lipschitz constraint. It then computes a per-sample certified reuse radius from the local classification margin and the spectral norm of the classification head. At runtime, a cached result is reused only when the query feature falls inside the certified reuse ball; otherwise, the query falls back to $\mathtt{\text{MainNet}}$. Thus, cache hits are transformed from empirical threshold tests into geometric certification decisions with explicit theoretical boundaries. Across standard image classification tasks like CIFAR, Tiny-ImageNet, and SVHN, $\mathtt{\text{LipCache}}$ achieves a measured speedup of up to $1.65\times$ with limited end-to-end accuracy degradation, while all accepted cache hits satisfy the $\mathtt{\text{GuardNet}}$-side certified-consistency condition. Furthermore, an enhanced $\mathtt{\text{GuardNet}}$ training recipe substantially improves cache hit rates in the Tiny-ImageNet multi-class extension while maintaining a certified-consistency rate of $100\mathrm{\%}$. These results demonstrate that per-sample certified reuse can reduce main-model fallback while preserving theoretical consistency, providing a feasible approach to reliable cache-assisted inference at the edge.