EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
We propose EngramEdit for decoupled knowledge updates through conditional memory.
Key points
- Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation.
- Beyond model scaling, this architecture has demonstrated the potential to decouple factual knowledge storage from general-purpose computation, offering a promising route to updating factual knowledge while keeping the Transformer backbone fixed.
- Realizing this potential is challenging because different expressions of a fact may activate different n-gram embeddings, while updating shared embeddings can unintentionally change the model's predictions about other facts.
- EngramEdit first computes target memory representations that make the model predict the updated fact across multiple expressions.
Sources (1)
- [1]EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional MemoryarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 05:58 PM
We propose EngramEdit for decoupled knowledge updates through conditional memory.
Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 7, 2026Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning
- Oct 6, 2026EmbeddingGemma 2: an open, lightweight multimodal embedding model
- Oct 5, 2026LiquidAI/d1-omni-600M
- Oct 5, 2026MC-Sparse: Deconstructing and Closing the Dense-Sparse Attention Gap in Diffusion Transformers
- Sep 30, 2026Cloudflare/clef-flash
- Sep 30, 2026huggingface/transformers v5.18.0: Release 5.18.0