AGAR: a reinforcement learning substrate for LLM program evolution

Key points

• Reinforcement learning already has an estimator for each.1
• We formalize it as a Markov decision process whose action is the modular prefix the model is conditioned on, rather than the program it emits.1
• AGAR (Algorithm Generation As RL) provides the resulting substrate: any estimator can be replaced or switched off without changing the controller, making the transfer auditable one mechanism at a time, with no gradient training of the backend model.1
• Across 19 tasks, two backends, and three seeds under one harness, AGAR improves on the stronger of two published baselines on most tasks, with gains concentrated in the competitive-programming family.1

Sources (1)

• [1]AGAR: a reinforcement learning substrate for LLM program evolution
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 11:22 PM

  Given a task and an evaluator, a language model can rewrite a candidate program while a search loop decides which rewrites survive, offering a practical route to algorithm discovery.

  Reinforcement learning already has an estimator for each.

Extractive summary: sentences quoted from the sources.