Double-Take: Improving Capacity, Binding, and Selectivity In Long-Context Within Bidirectional Memory
We introduce Double-Take (DT), which repairs these failures with multi-step unpinned updates and cross-position mixing, and adds bidirectional change-sensitive writes for selective retention.
ProofPaper ↗
Key points
- Processing a long sequence does not guarantee that distant information remains usable.
- As context grows, a fixed-size memory must retain many associations, connect information across positions, and preserve relevant structure without being overwhelmed by everything it observes.
- We study this problem in large-chunk test-time training (TTT) and identify two basic failures: single-step writes accumulate interference, limiting capacity, while self-associative writes cannot form cross-position bindings.
- Trained only on $32\,kb$ genomic contexts, DT reaches $0.927$ AUROC at $131\,kb$ and $0.856$ near $1\,Mb$, while transferring zero-shot from human to mouse.
Sources (1)
- [1]Double-Take: Improving Capacity, Binding, and Selectivity In Long-Context Within Bidirectional MemoryarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 06:31 AM
We introduce Double-Take (DT), which repairs these failures with multi-step unpinned updates and cross-position mixing, and adds bidirectional change-sensitive writes for selective retention.
Processing a long sequence does not guarantee that distant information remains usable.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse
- Oct 9, 2026RAMP: Recurrent Adaptive Mixed Precision for Efficient Prefill in Recurrent-Depth Language Models
- Oct 8, 2026[AINews] not much happened today
- Oct 8, 2026Read What Matters: Query-Adaptive Quantization for KV Caches
- Oct 8, 2026REMORY: Learning Residual Memory for Context Compaction
- Oct 7, 2026Dual-QK: Sharp Queries and Flat Keys for Prunable 2-bit KV Caches