ReplaySSM: Cache SSM Inputs, Not State
We introduce ReplaySSM, which unrolls the recurrence to represent the state as a checkpoint and a buffer of recent SSM inputs.
ProofPaper ↗
Key points
- State-space models (SSMs) maintain a fixed-size recurrent state, but reading and writing it at every decoding step can bottleneck hybrid models.
- This eager state management causes large state I/O with low arithmetic intensity, leaves speculative decoding no cheap rollback, and feeds state quantization error into subsequent recurrent updates.
- Keeping recent inputs also makes speculative rollback a buffer-pointer operation and enables output-only verification without materializing intermediate states.
- We implement ReplaySSM in vLLM for Mamba-2 and Gated DeltaNet.
Sources (1)
- [1]ReplaySSM: Cache SSM Inputs, Not StatearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 03:59 PM
We introduce ReplaySSM, which unrolls the recurrence to represent the state as a checkpoint and a buffer of recent SSM inputs.
State-space models (SSMs) maintain a fixed-size recurrent state, but reading and writing it at every decoding step can bottleneck hybrid models.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026CNN-Transformer with Sigmoid Label for High-Resolution Sub-Degree DOA Estimation
- Oct 9, 2026Surfing the Activation Wave: Analyzing the Information and Geometry of LLM Hidden States for Speculative Decoding
- Oct 8, 2026mistralai/Voxtral-Mini-4B-Realtime-Arabic
- Oct 8, 2026Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers
- Oct 8, 2026ConwayResearch/Underdog-Saluki-27B-1.0
- Oct 8, 2026Contextual Correction of Parallel Draftsfor Speculative Decoding