LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs

Key points

• A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage.1
• To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired.1
• Each edit is first stored at low rank as a cheap sketch.1
• Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA-3-8B, Mistral-7B, and Qwen2.5-7B, LadderEdit tracks exact LoRA storage at 5.2x less memory and remains effective at 50,000 sequential edits.1

Sources (1)

• [1]LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 03:20 AM

  Lifelong editing of LLMs requires storing thousands of edits after acquisition.

  A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage.

Extractive summary: sentences quoted from the sources.