DIP: Dynamic In-Context Planner For Diffusion Language Models

Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. However, due to the bidirectional attention mechanism, DLMs incur substantial computational cost as context length increases. This work addresses this issue with a key discovery: unlike the sequential generation in autoregressive language models (ARLMs), the diffusion generation paradigm in DLMs allows $\mathit{\text{efficient}}\mathit{\text{dynamic}}\mathit{\text{adjustment}}\mathit{\text{of}}\mathit{\text{the}}\mathit{\text{context}}$ during generation. Building on this insight, we propose $\mathbf{\text{D}}$ynamic $\mathbf{\text{I}}$n-Context $\mathbf{\text{P}}$lanner (DIP), a context-optimization method that dynamically selects and inserts in-context examples during generation, rather than providing all examples in the prompt upfront. Results show DIP maintains generation quality while achieving up to 12.9$\times$ inference speedup over standard inference and 1.17$\times$ over KV cache-enhanced inference.