MLIPS: Multi-Layer Indexer Pool Sharing

Sparse attention limits the keys used by each query, but finding those keys can still require a full-prefix scan at every layer. We introduce $\mathbf{\text{MLIPS}}\mathbf{\text{(Multi-Layer}}\mathbf{\text{Indexer}}\mathbf{\text{Pool}}\mathbf{\text{Sharing)}}$, a training-free method that shares candidate search while retaining each consumer’s own scoring and selection. A producer publishes a top-$M$ pool larger than the attention budget $K$; subsequent layers select their own top-$K$ from that pool. Measurements across three sparse-attention model families show substantially higher coverage from broad pools than overlap between final selections. We implement layerwise sharing, persistent decode reuse, and a coarse producer in vLLM for DeepSeek-V4-Flash. Across four long-context suites, three consumer layers per producer give $1.14$–$1.40\times$ speedups and $14.8$–$27.4\mathrm{\%}$ lower GPU energy, with observed accuracy changes of $-1.49$ to $+0.29$ percentage points in single runs. More extensive sharing exposes task-dependent quality losses. Sampled prefill traces also reveal recurring selected KV positions across query groups and layers. We characterize this shared working-set structure and use it to motivate Top-$G$, an exploratory bounded pool construction for prefill. Its deployment, task accuracy, and serving benefits remain open.