Agent Plasticity: Measuring Self-Improvement Through Experience

Key points

• AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns.1
• Evaluating self-improvement requires answering three questions: does future performance improve and generalize beyond the interactions that enabled learning; how efficiently are new capabilities acquired; and where does the self-improvement process break down?1
• To answer these questions, we study self-improvement in a controlled setting where agents amortize past experience into reusable artifacts that are inherited by future instances.1
• Evaluating self-improving agents requires measuring not only what they can do, but how effectively they become better through experience.1

Sources (2)

• [1]Agent Plasticity: Measuring Self-Improvement Through Experience
  Hugging Face Daily Papers · Oct 6, 12:00 AM

  We introduce agent plasticity, the efficiency with which an agent converts experience into gains in future held-out performance.

  AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns.

• [2]Agent Plasticity: Measuring Self-Improvement Through Experience
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 05:58 PM · same content

Extractive summary: sentences quoted from the sources.