DP-MIND: Differentially private memory-guided Long context Inference

Key points

• LLM-as-a-Service has become a dominant paradigm for accessing large language models without requiring local deployment.1
• Differential privacy (DP) provides a principled approach to protecting such inputs, but existing DP inference methods are primarily designed for short- or moderate-context settings and struggle with million-token-scale inputs due to limited context capacity and severe utility degradation under DP perturbation.1
• DP-MIND enables private long-context inference by locally perturbing the input and processing it chunk by chunk with a compact memory that preserves task-relevant information across the sequence.1
• To make this memory robust to DP noise, we optimize the memory construction policy directly on perturbed inputs using reinforcement learning, guided by both final answer correctness and memory consistency.1

Sources (1)

• [1]DP-MIND: Differentially private memory-guided Long context Inference
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 07:25 AM

  In this paper, we propose DP-MIND, a memory-guided framework that scales differentially private inference to million-token contexts.

  LLM-as-a-Service has become a dominant paradigm for accessing large language models without requiring local deployment.

Extractive summary: sentences quoted from the sources.